We evaluated Meta’s new Muse Spark 1.1 across our benchmark suite.

• It debuts at #4 on the Vals Index (68.41%), narrowly ahead of GPT 5.5 (67.95%). At $0.50/test, it is the lowest-cost model in the top 10; it also averaged 388.5 seconds on the index, roughly 3× faster than each of the top three models.

• It takes #2 on Finance Agent v2 (57.21%), just 0.65 points behind Gemini 3.5 Flash.

• The largest generational gain is on Vibe Code Bench. Compared with Muse Spark, Muse Spark 1.1 rises from #43 to #5 and improves by 52.48 points (19.67% → 72.16%).

• Muse Spark 1.1 sets new highs on domain-specific work: #1 on MedScribe (88.89%), #1 on TaxEval v2 (79.72%), and #1 on Harvey’s Legal Agent Benchmark (20.00%). On MedScribe, it is nearly twice as fast as the #2 model.

The model has a 1M-token context window and supports up to 256k output tokens. Evaluations were run with reasoning effort set to “xhigh” and default temperature and top-p.

Congrats to the Meta team on the release!

We evaluated xAI’s new Grok 4.5 across our benchmark suite.

• It ranks #6/32 on the Vals Index (65.30%), and takes #1 on Harvey’s Legal Agent Benchmark (12.92%).

• Coding is a highlight: #3 on SWE-bench Verified (86.60%), #7 on LiveCodeBench (87.35%), #7 on Vibe Code Bench (69.00%), and #9 on Terminal-Bench 2.1 (67.79%).

• It also posts #9 on Finance Agent v2 (48.35%) and #4 on GPQA Diamond (92.93%).

Evaluations were run via xAI’s Grok API, with a temperature of 0.7, top_p set to 0.95, and reasoning effort set to “high”.

Congrats to the xAI team on the release!

We’ve released the Excel Modeling Benchmark (EMB), a private benchmark that tests whether agents can build the complex Excel financial models used in investment banking and private equity — LBOs, DCFs, M&A, and more — graded against expert-authored gold models.

• Claude Opus 4.8 leads at 69.4%, ahead of Claude Sonnet 5 (66.3%) and GPT 5.5 (64.5%).
• Numerical accuracy is the bottleneck: the leading model passes 87% of formula checks but only 61% of number checks — structurally sound models whose computed values drift from the reference.
• Claude Opus 4.8 is the most accurate at ~$12 per task, while Claude Sonnet 5 ranks second but is the priciest of all at $15.44; MiMo V2.5 Pro stays above 50% at just $0.22.

• We evaluated Anthropic’s new Claude Sonnet 5 across our benchmark suite. It comes in at #3 on the Vals Index (68.61%), behind only Claude Fable 5 (75.15%) and Claude Opus 4.8 (70.36%), and narrowly ahead of GPT 5.5 (67.95%).

• It’s a substantial generational step: +8.5 points on the Vals Index over Claude Sonnet 4.6 (60.07%) — and almost all of that gain comes from coding.

• The models main strength is in coding. Compared with Sonnet 4.6, Claude Sonnet 5 jumps +30.7 points on Vibe Code Bench (56.22% → 86.90%) and +17.2 points on Terminal-Bench 2.1 (57.30% → 74.53%), while SWE-bench Verified actually slips slightly (77.45% → 75.49%). The gains are concentrated in sustained, multi-step agentic coding rather than one-shot patch generation.

• Outside coding it’s a quieter step. Sonnet 5 posts 67.95% on CorpFin v2 (+1.3 vs Sonnet 4.6) and 51.98% on Finance Agent (+0.9). Its index score clears last-generation Claude Opus 4.7 (66.10%).

We noticed a small but noticeable number of refusals on CorpFin v2 — 15 of 858 tasks, mostly flagged as “bio.”

The model has a 1M-token context window. Evaluations were run with compute effort set to “max” on all benchmarks except Terminal-Bench 2.1 (run at “high”), 128k max output tokens, and default temperature and top-p. Terminal-Bench 2.1 was averaged over 3 runs (71.9%–78.7%).

Congrats to the Anthropic team on the release!