We’ve finished our evaluation of GLM 5.1 and it’s now the #1 open-weight model on the Vals Index, unseating Kimi K2.5, and ranking #6 overall on the index.

Key takeaways:

• GLM 5.1 leads open-weight models on coding benchmarks, ranking #1 open-weight on Vibe Code Bench, Terminal Bench 2, and SWE-bench Verified.
• It’s also #1 on our in-house Finance Agent (v1.1).
• It ranks #1 on MedCode, but is lower on TaxEval (#8 open-weight) and CaseLaw (#13 open-weight).
• It comes in at $0.22/test on the index — more expensive than Kimi K2.5 ($0.12/test), but cheaper than GLM 5, and far below models like Claude Sonnet 4.6 ($0.80) or GPT 5.4 ($0.67).

Eval settings: temperature=1, top_p=0.95, 131k max output tokens, run via the native GLM API. Context window: 200k tokens.

Congrats to the team at z.AI on the release!

We added Trinity Large Thinking to the Vals Index. Highlights:

• Among open-weight models on the index, it ranks #10 overall (42.6% accuracy).
• Strongest pillar is law: #6 among open-weight models on Case Law (v2) (57.9%).
• Weaker on finance and coding relative to peers: near the bottom of the open-weight set on Finance Agent, Corp Fin (v2), SWE-bench, and Terminal-Bench 2.0.

The model was tested with temperature=0.3 and top_p=0.95, via Arcee AI’s API endpoint.

Congratulations to Arcee AI on the release!

We have migrated our SWE-bench evaluation harness from SWE-agent to Mini-SWE-agent, a minimal, bash-tool-only agent. Models on the Vals Index have been re-evaluated under the new harness and results have been updated accordingly.

Why we switched

SWE-agent provides models with a rich set of custom tools (file navigation, search, edit/insert) on top of bash. While this can boost scores, it also means the harness itself is doing meaningful work — making it harder to isolate the model’s raw capabilities. Different models also interact with these specialized tools in very different ways, introducing a confounding variable.

Our new harness gives models a single tool: bash. This is a simpler, more transparent setup that relies on the model to navigate, search, edit, and solve tasks using standard command-line tools.

What changed

• Harness: SWE-agent (multi-tool) replaced by mini-swe-agent (bash only)
• Models: All models re-evaluated from scratch
• Results: Most models saw score small changes of 1-5 percentage points in either direction. A few models saw larger shifts reflecting how well they adapt to a bash-only workflow versus SWE-agent’s specialized tooling.

We evaluated MiniMax-M2.7 across our benchmarks.

• It placed #12 (60.14%) on the Vals Index, comparable to both GLM 5 and Claude Sonnet 4.5 (Thinking).
• On the Vals Index, MiniMax-M2.7 marks a significant jump over both MiniMax-M2.5 and MiniMax-M2.1, a rapid rate of improvement over the last three months.
• MiniMax-M2.7 had a strong performance on coding benchmarks, ranking #10 on SWE-Bench beating out Gemini 3 Pro (11/25) and #12 on Terminal-Bench 2, beating out its predecessor MiniMax-M2.5.
• Performance was weak on our Formal Math benchmark scoring just 3% on Proof Bench ranking at #23.
• On legal and finance tasks, results were mixed - it scored #19 on our Finance Agent v1.1 and #24 on our Corp Fin benchmark.

The model was tested with temperature=1 and top_p=0.95, via the Anthropic-compatible MiniMax API endpoint.

Congratulations to MiniMax on the release!