We evaluated Gemini 3.1 Pro Preview (02/26) on our Vals Index. Here are the key takeaways:

• Gemini 3.1 Pro Preview (02/26) demonstrates strong performance on Terminal Bench 2, achieving 67.4% accuracy and taking first place by a distance, beating out the 2nd-ranked model, Claude Sonnet 4.6, at 59.55%.
• The model shows dramatic improvement compared with Gemini 3 Pro (11/25) on Case Law (v2)**, jumping from rank #50 (53.4% accuracy) to rank #11 (65.6% accuracy)—a 12 percentage point improvement.
• On Corp Fin (v2)‘s Vals Index split, the model achieves 65.7% accuracy. The model ties for 6th place on the Finance Agent subset (achieving 58.9% accuracy), matching Gemini 3 Pro (11/25).
• On SWE-bench, the model achieves 69.6% accuracy, demonstrating solid performance on software engineering tasks, but falling behind Gemini 3 Pro (11/25).

We are evaluating Gemini 3.1 Pro Preview (02/26) on our full suite of benchmarks and will be sharing updates soon!

We’re releasing MedScribe, a new benchmark evaluating AI systems on clinical SOAP note generation, built in collaboration with Harvard Medical School and Protege.

• GPT 5.1 takes the lead, exceeding 88% accuracy. The Claude family also demonstrates strong performance.
• Results suggest models are capable of producing structurally sound, clinically relevant medical notes and can meaningfully assist scribes in their workflow.
• Most models perform marginally worse on the Plan section, where even small errors can affect care continuity and billing accuracy.

We’re releasing MedCode, a new benchmark evaluating AI systems on medical coding from realistic patient documentation, built in collaboration with Harvard Medical School and Protege.

• Models barely exceed 55% accuracy on this critical healthcare task, with Gemini 3 Flash (12/25) taking the lead at 55.9%.
• Models perform better on physical conditions like diabetes and hypertension but struggle significantly with mental health diagnoses.
• Even at the category level, models frequently miss diagnoses entirely rather than mislabelling specificity.

As we finish evaluating Qwen 3.5 Plus, here are our key takeaways:

• The model performs well on knowledge benchmarks, ranking #6 on GPQA, #11 on MedQA and #9 on MMLU Pro, all of which are first place ranks among Open Weight models.
• Qwen 3.5 Plus has a varied performance on our private benchmarks, ranking #30 on both the Mortgage Tax and SAGE benchmarks, but ranking #8 (and #1 among Open Weight models) on Finance Agent v1.1.
• It loses out on performance due to having a highly sensitive content filter and havng issues following a strict output format. This leads to low performance on the CaseLaw (v2) and MMMU benchmarks.