DeepSeek V3.2 (Nonthinking) has just claimed first place on our Open-Weight Vals Index at a third of the price of leading competitors!

• Both DeepSeek V3.2 (Nonthinking) and DeepSeek V3.2 (Thinking) lead the way with competitive performance on SWE-Bench and our proprietary Finance Agent Benchmark.
• Surprisingly, we found that DeepSeek V3.2 (Nonthinking) generally outperformed DeepSeek V3.2 (Thinking) - see our comparison page for details
• Both models constitute a significant improvement over DeepSeek V3.1

That said, neither model is truly SOTA: the best placement either achieves is ninth on Terminal Bench. With Christmas coming up, we look forward to further innovations from DeepSeek!

GPT 5.2 is the new state of the art on our Vals Index, showing strong performance across domains, especially coding.

Most impressive was the model setting a new state-of-the-art on Vibe Code Bench by an extremely large margin, from 24.6% to 35.6%. It also got first on our IOI, Terminal Bench, and SWE Bench.

This performance improvement does come with an increased cost across the board - the model is priced at $1.75 / $14, compared to $1.25 / $10 for its predecessor. It also tends to see increased token usage, especially on longer running agentic tasks. With the “high” or the new “xhigh” reasoning modes enabled, you may also see very long response times - for particularly tricky questions, it would think for more than 30 minutes.

Overall, the model will be a powerhouse for users seeking strong performance and reliability, particularly on complex reasoning and coding tasks.

We’ve evaluated GPT 5.1 Codex Max on our coding benchmarks. It boasts a +9.5% performance boost on VibeCodeBench (#3), +1% performance on SWE Bench (#4), and a slight regression on Terminal Bench.

This is not the fastest model in the shed. It’s 3x slower on SWE Bench, 10x on VCB. This is a model for when you need the absolute best, not when you need it quickly.

Correlation isn’t causation… …but the more suffixes we tack onto these models, the longer they seem to take. Excited to test 5.25 Codex Max Ultra High Supreme Edition

The model was run with reasoning high and verbosity medium through OpenAI’s API. Results on IOI and LCB will be released soon!

We found that Mistral Large 3 struggles across domains:

• We found a variety of difficulties with agentic use-cases involving tool-calling. We’re working with the Mistral team to address one such issue.
• The model places second-to-last on our Vals Multimodal Index, ahead of Llama 4 Maverick.

However, there are a few caveats:

• the model represents a meaningful improvement over predecessors like Magistral Medium 1.2 (09/2025) and Mistral Medium 3.1 (05/2025).
• The model is open-weight and performs comparably to other open-weight models like GPT OSS 120B.

We look forward to further improvements from Mistral to come!