We’ve released Finance Agent v2, the next generation of our benchmark for evaluating AI agents on realistic financial analyst tasks.

• Nine analytical categories spanning retrieval, market analysis, comparables, precedents, adjustments, earnings, disclosure, and financial modeling.
• Severity-weighted partial credit with dealbreaker gating: a question receives no credit when a load-bearing fact is wrong, regardless of how much of the surrounding answer is correct.
• Every model is evaluated three times on the full benchmark; reported scores are the mean across runs.

Compared to v1.1, scores fall ~14 points on average across models evaluated on both versions, with the steepest drop in Financial Modeling — where multi-step arithmetic and finance-specific conventions are least forgiving under the new rubric.

GPT 5.5, Claude Opus 4.7, and Claude Sonnet 4.6 are the top three performers, separated by less than a point.

We have updated the Vals Index and the Vals Multimodal Index to better reflect the capabilities that matter most when deploying frontier models to real economic work.

Vibe Code Bench now joins the coding bucket alongside SWE-bench Verified and Terminal Bench 2.

The CaseLaw benchmark had become saturated, and was no longer providing useful differentiation between models. Consequently, it was removed from both indexes.

We evaluated Mistral Medium 3.5 on our suite of benchmarks. Here are the key takeaways:

• Mistral’s new reasoning-mode Medium 3.5 lands at #32 of 46 on the Vals Index (52.77%), and #10 of 18 among open-weight models.
• It is a sizable jump over the prior Mistral Large 3 across most of the suite: +28 points on Finance Agent (46.1% vs 18.1%), +29 points on the SWE-bench Verified subset of the Vals Index (64.7% vs 35.3%), +21 points on Terminal-Bench 2.0 (30.3% vs 9.0%), and +13 points on SAGE (37.6% vs 24.6%).
• One regression worth flagging: Case Law (v2) drops 17 points (44.2% vs 61.4%).

All evals were run via the Mistral API at temperature 0.7, top-p 0.95, with reasoning_effort: high and an 80k max output budget.

Results for xAI’s Grok 4.3 are now available:

• Grok 4.3 places #13 of 46 on the Vals Index at 62.6%.
• It takes #1 on CorpFin (v2) (68.5%) and #1 on Case Law (v2) (79.3%).
• On Vibe Code Bench, it scores 19.4% — a +15-point improvement over its predecessor.

We evaluated Grok 4.3 with a temperature 0.7, top_p 0.95, and default top_k. The model supports a 1M-token context window, and had an average latency of 584.24 seconds per test at $0.38 per test on the index.