We evaluated OpenAI’s new GPT-5.6 Sol and GPT-5.6 Terra across our benchmark suite.

• Sol ranks #2 on the Vals Index (72.63%) and the Vals Multimodal Index (72.19%).

• Agentic coding is the headline: Sol takes #1 on SWE-bench Verified (96.20%) and Terminal-Bench 2.1 (85.77%). It also scores 80.50% (#4) on Vibe Code Bench and takes #1 on ProofBench (77.00%).

• Sol also leads CyberBench (88.14%), the Excel Modeling Benchmark (72.34%), and Legal Research Bench (48.08%).

• Terra scores 65.14% on the Vals Index and 65.07% on the Vals Multimodal Index, with strong results on ProofBench (71.00%, #3), Legal Research Bench (40.87%, #4), and Terminal-Bench 2.1 (73.41%, #7).

We evaluated Meta’s new Muse Spark 1.1 across our benchmark suite.

• It debuts at #4 on the Vals Index (68.41%), narrowly ahead of GPT 5.5 (67.95%). At $0.50/test, it is the lowest-cost model in the top 10; it also averaged 388.5 seconds on the index, roughly 3× faster than each of the top three models.

• It takes #2 on Finance Agent v2 (57.21%), just 0.65 points behind Gemini 3.5 Flash.

• The largest generational gain is on Vibe Code Bench. Compared with Muse Spark, Muse Spark 1.1 rises from #43 to #5 and improves by 52.48 points (19.67% → 72.16%).

• Muse Spark 1.1 sets new highs on domain-specific work: #1 on MedScribe (88.89%), #1 on TaxEval v2 (79.72%), and #1 on Harvey’s Legal Agent Benchmark (20.00%). On MedScribe, it is nearly twice as fast as the #2 model.

The model has a 1M-token context window and supports up to 256k output tokens. Evaluations were run with reasoning effort set to “xhigh” and default temperature and top-p.

Congrats to the Meta team on the release!

We evaluated xAI’s new Grok 4.5 across our benchmark suite.

• It ranks #6/32 on the Vals Index (65.30%), and takes #1 on Harvey’s Legal Agent Benchmark (12.92%).

• Coding is a highlight: #3 on SWE-bench Verified (86.60%), #7 on LiveCodeBench (87.35%), #7 on Vibe Code Bench (69.00%), and #9 on Terminal-Bench 2.1 (67.79%).

• It also posts #9 on Finance Agent v2 (48.35%) and #4 on GPQA Diamond (92.93%).

Evaluations were run via xAI’s Grok API, with a temperature of 0.7, top_p set to 0.95, and reasoning effort set to “high”.

Congrats to the xAI team on the release!

We’ve released the Excel Modeling Benchmark (EMB), a private benchmark that tests whether agents can build the complex Excel financial models used in investment banking and private equity — LBOs, DCFs, M&A, and more — graded against expert-authored gold models.

• Claude Opus 4.8 leads at 69.4%, ahead of Claude Sonnet 5 (66.3%) and GPT 5.5 (64.5%).
• Numerical accuracy is the bottleneck: the leading model passes 87% of formula checks but only 61% of number checks — structurally sound models whose computed values drift from the reference.
• Claude Opus 4.8 is the most accurate at ~$12 per task, while Claude Sonnet 5 ranks second but is the priciest of all at $15.44; MiMo V2.5 Pro stays above 50% at just $0.22.