We just evaluated GPT 5.1 on all benchmarks. Our findings are below:

• GPT 5.1 takes second place on the Vals Index, behind the reigning champion, Claude Sonnet 4.5 (Thinking).
• GPT 5.1 shines on agent benchmarks, placing first on our Finance Agent Benchmark.
• GPT 5.1 also excels at competitive coding, placing second on both LiveCodeBench and our IOI Benchmark.
• GPT 5.1 performs comparably to GPT 5, but much more cheaply and quickly - we consider it a clear leader on Pareto frontier between performance and cost.

We evaluated Kimi K2 Thinking on all benchmarks, finding strong performance on coding benchmarks like SWE-bench. We otherwise found middle-of-the-pack performance on both public and private benchmarks.

In comparison to Kimi K2 Instruct, we found a modest improvement across most industries, with the exception of legal and finance. For instance, the Instruct model outperforms the thinking model by nearly 5% on our TaxEval benchmark.

Overall, we found Kimi K2 Thinking to be a strong successor to Kimi K2 Instruct, but doesn’t live up to the twitter hype of “best model on the market”.

We just evaluated Kimi K2 Thinking on our Vals Index! We found it to improve upon the already solid performance of Kimi K2 Instruct, placing 2nd in our open-weight category.

Kimi K2 Thinking especially shines on agentic use-cases requiring iterative tool calling, like our Finance Agent Benchmark. It performs similarly to Kimi K2 Instruct on other benchmarks.

One meaningful downside of the model is its latency - on average, Kimi K2 Thinking takes almost five times as long as GLM 4.6, the leader on our open-weight Vals Index.

Overall, we see Kimi K2 Thinking as an important contribution to the open-source ecosystem.

We’re excited to introduce the Multimodal (image-based) variant of our Vals Index ! It extends upon the original Vals Index with two of our private multimodal benchmarks:

• MortgageTax, which tests models’ ability to read and understand tax certificates
• SAGE, which tests models’ ability to grade handwritten undergraduate student work

As with our Vals Index, Claude Sonnet 4.5 (Thinking) tops the Vals Multimodal Index leaderboard, followed closely by GPT 5 and Claude Haiku 4.5 (Thinking). Price-conscious consumers might turn to Haiku or GPT 5 Mini, while Gemini 2.5 Pro and Grok 4 lag behind.

For our initial release, we only include closed-source models from 4 major providers: OpenAI, Anthropic, Google, and xAI. In the future, we hope to add open-source multimodal models like Qwen 3 VL Plus.