We launched the Web Search Index, which isolates the effect of the web-search tool itself: each model runs the same legal-research and finance-analysis tasks with its own native provider search versus an independent web-search tool (Exa), holding the model fixed.

• Across all four models tested (Claude Fable 5, GPT-5.6 Sol, Gemini 3.5 Flash, and Grok 4.5), swapping in Exa improved overall accuracy — by roughly 1.5 to 4.3 points.

• The gains concentrate in finance analysis, where every model scored higher with Exa — for example, Grok 4.5 rose +9.8 points (32.1% → 41.9%) and Claude Fable 5 +8.8 points (44.8% → 53.6%).

• Legal research was a wash: native search led for Claude Fable 5 and Grok 4.5, while Exa led for Gemini 3.5 Flash and GPT-5.6 Sol — so the better tool depends on the model.

• Total cost is dominated by model inference: Exa’s flat per-search fee is a negligible share of per-task cost, and native provider search is cheaper on average.

See the full leaderboard, per-domain breakdowns, and search-behavior analysis on the Web Search Index.

We evaluated Google’s Gemini 3.5 Flash Lite on the Vals Index and across our benchmark suite.

• Gemini 3.5 Flash Lite scores 51.00% on the Vals Index, placing #25 of 39 models overall.

• Its Index component results include 68.63% on the SWE-bench Verified subset, 62.24% on the CorpFin v2 subset, and 46.63% on the Finance Agent v2 subset.

• The model scores 32.77% on the Vibe Code Bench Index subset and 50.19% across three full trials of Terminal-Bench 2.1.

We evaluated Gemini 3.5 Flash Lite with temperature 1, high reasoning effort, and up to 65k output tokens. The model supports a 1M-token context window, multimodal inputs, and tool calling.

Results are now available across 25 benchmarks. On the full 500-task SWE-bench Verified evaluation, Gemini 3.5 Flash Lite scores 75.00%; the Vals Index uses the 68.63% subset result above.

We evaluated Google’s Gemini 3.6 Flash on the Vals Index and proprietary benchmarks.

• Gemini 3.6 Flash scores 62.43% on the Vals Index, placing #14 of 38 models overall and finishing within 0.32 points of Gemini 3.5 Flash.

• Gemini 3.6 Flash ranks #1 on the CyberBench Patch track at 84.75%, which measures fixing real-world open-source vulnerabilities.

• Its coding results include 77.45% on the SWE-bench Verified Vals Index subset and 57.88% on the Vibe Code Bench Vals Index subset, improvements of 1.96 and 3.17 points over Gemini 3.5 Flash, respectively.

• Gemini 3.6 Flash scores 73.78% across three full trials of Terminal-Bench 2.1, placing #8 of 43 models. It also places #7 of 74 models on MedCode, scoring 53.15%.

We evaluated Gemini 3.6 Flash with temperature 1, high reasoning effort, and up to 65k output tokens. The model supports a 1M-token context window, multimodal inputs, and tool calling.

Congrats to the Google team on the release!

We evaluated Moonshot AI’s Kimi K3 on the Vals Index.

• Kimi K3 scores 74.70% on the Vals Index, placing #2 of 38 models overall.

• Its strongest component results are 95.10% on the SWE-bench Verified Vals Index subset and 91.27% on the Vibe Code Bench Vals Index subset.

• Kimi K3 scores 80.90% across three full trials of Terminal-Bench 2.1, 72.61% on CorpFin v2, and 55.88% on the Finance Agent v2 Index subset.

We evaluated Kimi K3 with temperature 1, up to 262k output tokens, and max-effort reasoning. Kimi K3 supports a 1M-token context window, image inputs, and tool calling.

Congrats to the Moonshot AI team on the release!