# Vals AI

> Independent AI benchmark results, model comparisons, research, and evaluation tools.

Vals AI publishes benchmark methodology and results for AI systems across legal, finance, coding, medical, mathematics, and other domains. Use the published benchmark and model pages as the source of record.

## Primary pages

- [Home](https://www.vals.ai/home)
- [Benchmarks](https://www.vals.ai/benchmarks): every current benchmark leaderboard
- [Model comparisons](https://www.vals.ai/comparisons)
- [Compare any two models](https://www.vals.ai/comparison)
- [Methodology](https://www.vals.ai/methodology)
- [About](https://www.vals.ai/about)

## Benchmarks

### Index

- [Vals Index](https://www.vals.ai/benchmarks/vals_index): A single measure of AI's potential economic impact — agentic model performance across finance, coding, legal, and tax tasks, weighted by each sector's share of U.S. GDP.
- [Vals RSI Index](https://www.vals.ai/benchmarks/rsi_index): Can a model do the research that builds the next model? Autonomous LLM R&D scored against published, human, and frontier-model references.
- [Vals Web Search Index](https://www.vals.ai/benchmarks/web_search): Comparing native provider search against independent web-search tools on legal-research and finance-analysis tasks
- [Vals Science Index](https://www.vals.ai/benchmarks/science_index): A single measure of AI's ability to do science — agentic model performance on biology data investigation, mechanism discovery, and research workflows, weighted equally.

### Legal

- [Legal Research Bench](https://www.vals.ai/benchmarks/legal_research): Evaluating agents on legal research tasks across diverse areas of US law
- [Harvey's Legal Agent Benchmark](https://www.vals.ai/benchmarks/hlab): Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools.
- [LegalBench](https://www.vals.ai/benchmarks/legal_bench): Evaluating language models on a wide range of open source legal reasoning tasks.

### Finance

- [Finance Agent v2](https://www.vals.ai/benchmarks/fabv2): Evaluating agents on core financial analyst tasks
- [Excel Modeling Benchmark](https://www.vals.ai/benchmarks/emb): Evaluating agents on Excel-based financial modeling tasks
- [Tax Agent Bench](https://www.vals.ai/benchmarks/tax_agent_bench): Evaluating agents on research-grade US corporate tax questions

### Healthcare

- [MedCode](https://www.vals.ai/benchmarks/medcode): Can models support the medical billing process?
- [MedScribe](https://www.vals.ai/benchmarks/medscribe): Can models support doctors with their administrative work?

### Math

- [ProofBench v1.1](https://www.vals.ai/benchmarks/proof_bench): Can models write math proofs that are formally verified?

### Science

- [Terminal-Bench-Science](https://www.vals.ai/benchmarks/terminal-bench-science): Research workflow tasks contributed by practicing scientists across five domains
- [BioMysteryBench](https://www.vals.ai/benchmarks/biomysterybench): Can agents solve agentic biology data investigations?
- [MysteryMechanism](https://www.vals.ai/benchmarks/mysterymechanism): Can agents rediscover sealed mathematical mechanisms through bounded experiments?
- [Robo-bench](https://www.vals.ai/benchmarks/robo_bench): Can a model control a robot arm and a humanoid in a real-time physics simulation, from motor commands and camera images alone? Most tasks are held out.

### Education

- [SAGE](https://www.vals.ai/benchmarks/sage): Student Assessment with Generative Evaluation

### Voice

- [VoiceCodeBench](https://www.vals.ai/benchmarks/voice-code-bench): How good are speech-to-text models at handling exact structured values in English workplace speech?

### Coding

- [Vibe Code Bench v1.1](https://www.vals.ai/benchmarks/vibe-code): Can models build web applications from scratch?
- [Vibe Code Bench 1-100](https://www.vals.ai/benchmarks/vcb-1-100): Can models extend a working web application across future user requests?
- [Code Migration](https://www.vals.ai/benchmarks/code-migration): Can language models reimplement working programs in another language?
- [ProgramBench](https://www.vals.ai/benchmarks/programbench): Can language models rebuild programs from scratch?
- [Terminal-Bench 4.0](https://www.vals.ai/benchmarks/terminal-bench-4): Frontier-difficulty terminal tasks across software, science, ML, operations, hardware, security, and media
- [SkillsBench](https://www.vals.ai/benchmarks/skillsbench): How important are skills for agents?
- [IOI](https://www.vals.ai/benchmarks/ioi): International Olympiad in Informatics
- [SWE-bench Multimodal](https://www.vals.ai/benchmarks/swebench-multimodal): Do AI Systems Generalize to Visual Software Domains?

### Cyber

- [CyberBench v1.1](https://www.vals.ai/benchmarks/cyber): Can AI agents find security bugs and patch them?
- [SRE Bench](https://www.vals.ai/benchmarks/srebench): Can AI agents work out what a real-world binary does without its source code?

### Games

- [CUA-bench](https://www.vals.ai/benchmarks/cua_bench): Can an AI agent play six commercial video games with only a keyboard and a mouse? Half of them are held out.
- [Time Horizon Index: KSP](https://www.vals.ai/benchmarks/time_horizon_index): Can an AI agent build and run a space program in Kerbal Space Program?
- [Poker Agent](https://www.vals.ai/benchmarks/poker_agent): Which model can make the most money playing poker?

### Social Mobility

- [Public Benefits Bench v1.1](https://www.vals.ai/benchmarks/public-benefits-bench): Can AI help people navigate SNAP benefits?

### Archived

No longer run on new models; earlier results stay published.

- [AIME](https://www.vals.ai/benchmarks/aime): Challenging national math exam given to top high-school students
- [CaseLaw v2](https://www.vals.ai/benchmarks/case_law_v2): Private question-answer benchmark over Canadian court-cases.
- [CorpFin v2](https://www.vals.ai/benchmarks/corp_fin_v2): A private benchmark evaluating understanding of long-context credit agreements
- [GPQA Diamond](https://www.vals.ai/benchmarks/gpqa): Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.
- [LiveCodeBench](https://www.vals.ai/benchmarks/lcb): Our Implementation of the LiveCodeBench benchmark
- [MATH 500](https://www.vals.ai/benchmarks/math500): Academic math benchmark on probability, algebra, and trigonometry
- [MedQA](https://www.vals.ai/benchmarks/medqa): Evaluating language model bias in medical questions.
- [MGSM](https://www.vals.ai/benchmarks/mgsm): A multilingual benchmark for mathematical questions.
- [MMLU Pro](https://www.vals.ai/benchmarks/mmlu_pro): Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.
- [MMMU Pro](https://www.vals.ai/benchmarks/mmmu): Multimodal Multi-task Benchmark
- [MortgageTax](https://www.vals.ai/benchmarks/mortgage_tax): Evaluating reading and understanding tax certificates as images
- [SWE-bench Verified](https://www.vals.ai/benchmarks/swebench): Solving production software engineering tasks
- [TaxEval v2](https://www.vals.ai/benchmarks/tax_eval_v2): A Vals-created set of questions and responses to tax questions
- [Vals Multimodal Index](https://www.vals.ai/benchmarks/vals_multimodal_index): Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLMs can have on the economy.

## Vals Index leaders (updated 2026-10-18)

- [1. Gemini 4 Argon](https://www.vals.ai/models/google_gemini-4-argon): 68.90%
- [2. Claude Sonnet 5.5](https://www.vals.ai/models/anthropic_claude-sonnet-5-5): 67.04%
- [3. Claude Opus 5.5](https://www.vals.ai/models/anthropic_claude-opus-5-5): 66.97%
- [4. Claude Fable 5.1](https://www.vals.ai/models/anthropic_claude-fable-5-1): 65.83%
- [5. Wonder Full](https://www.vals.ai/models/meta_muse-wonder-full): 64.57%
- [6. Claude Opus 5](https://www.vals.ai/models/anthropic_claude-opus-5): 63.67%
- [7. GPT-6 Astra](https://www.vals.ai/models/openai_gpt-6-astra): 63.13%
- [8. Claude Fable 5](https://www.vals.ai/models/anthropic_claude-fable-5): 61.39%
- [9. GPT-6.1 Sol](https://www.vals.ai/models/openai_gpt-6.1-sol): 61.15%
- [10. Wonder](https://www.vals.ai/models/meta_muse-wonder): 61.06%

## Research and updates

- [Blog](https://www.vals.ai/blogs)
- [How Far Will AI Assistants Go for a Refund They Can’t Get?](https://www.vals.ai/blogs/impossible-refund): Six AI assistants pursued an impossible £350 train refund in a simulated world, escalating to government officials, legal clinics and bank chargebacks.
- [Routers Are a New AI Primitive](https://www.vals.ai/blogs/routers-new-ai-primitive): Model routers need an independent, end-to-end evaluation of quality, cost, and latency on real work. We're building one.
- [How pro-China are Chinese models?](https://www.vals.ai/blogs/china-us-analysis): It's complicated.
- [Do Agent Teams Pay Off? A Case Study on Vibe Code Bench](https://www.vals.ai/blogs/multi-agent-vibe-code-bench): Do AI coding-agent teams pay off? Compare GPT 6 Sol and Claude Opus 5.5 on Vibe Code Bench, including cost, reasoning effort and performance.
- [An Evaluation of Inception's Mercury Decide](https://www.vals.ai/blogs/evaluation-of-mercury-decide): Compare Mercury Decide, Jev and frontier LLMs on claim verification and a LegalBench subset, with accuracy, cost, latency and calibration results.
- [Trust but Verify: Our first Public Sector Summit](https://www.vals.ai/blogs/trust-but-verify-public-sector-summit): Reflections from our first Public Sector Summit at STATION DC on youth wellbeing, AI diplomacy with China, and independent evaluation.
- [Two-Thirds of MiMo v2.6's Coding Tasks Leak the Answer. Are Models That Exploit This Misbehaving?](https://www.vals.ai/blogs/mimo-reward-hacking): MiMo v2.6 reward-hacking audit: in two-thirds of Xiaomi's released coding environments, the fix remains in Git history. See how MiMo exploits it.
- [An Independent Evaluation of TypeSafe's Jev](https://www.vals.ai/blogs/independent-evaluation-of-jev): TypeSafe's Jev vs frontier LLMs on claim verification and a LegalBench subset: compare accuracy, cost, latency and calibration in Vals AI's evaluation.
- [Two Room-Temperature Antiferromagnetic Semiconductor Candidates](https://www.vals.ai/blogs/room-temperature-magnetic-semiconductors): Explore two room-temperature antiferromagnetic semiconductor candidates found by Claude Opus 5.5 agents, with calculations, code and caveats.
- [Vals Web Search Index: Evaluating Search for Real Work](https://www.vals.ai/blogs/web-search-index): Why Vals AI built the Vals Web Search Index: evaluating search tools on legal and finance research while holding the model and agent harness fixed.
- [Has the Bitter Lesson Come for AI Detectors?](https://www.vals.ai/blogs/ai-detection-benchmark): Compare AI writing detectors and LLMs on private human and AI-written text. Explore detection accuracy, false positives and adversarial rewrites.
- [A Lean Proof of the Thomson Problem for Seven Electrons](https://www.vals.ai/blogs/thomson-n7-lean-proof): Explore a Lean proof of the Thomson problem for seven electrons, produced by Claude Sonnet 5.5 agents, with the proof code and verification checks.
- [Evaluating AI Safety in Teen Conversations](https://www.vals.ai/blogs/evaluating-ai-safety-in-teen-conversations): How safe are AI chatbots for teens? Explore findings from 648 simulated conversations as teens disclose more, resist advice and seek help.
- [A Faster Shortest Path Algorithm](https://www.vals.ai/blogs/faster-shortest-path-algorithm): How Claude Opus 5.5 agents developed C-HD, a shortest-path algorithm with a Lean proof. Explore the result, assumptions and verification.
- [AI Cheating is on the Rise](https://www.vals.ai/blogs/cheating-on-the-rise): An integrity audit across BioMysteryBench, Terminal-Bench 2.1, and SWE-bench Verified shows that cheating increasingly complicates evaluation.
- [Vals Environmental Impacts Report](https://www.vals.ai/blogs/vals-environmental-impacts-report): Explore estimates of LLM energy use, carbon emissions and water consumption on Vals Index tasks, with model comparisons and methodology.
- [Claude Fable 5.1 Solves the Cyphral Distich](https://www.vals.ai/blogs/fable-solves-cyphral-distich): Explore Claude Fable 5.1's proposed solution to Sir Thomas Urquhart's Cyphral Distich, a centuries-old cipher, with the reasoning behind the result.
- [Series A: Always a Higher Peak](https://www.vals.ai/blogs/series-a): Vals AI announces a $40M Series A at a $400M valuation led by a16z, alongside new releases including Vals Smith, Frontier Risk Benchmarks, and Vals 2.0.
- [Child AI Safety](https://www.vals.ai/blogs/child-ai-safety): How Vals AI is developing youth-specific AI safety evaluations with clinicians and researchers to assess chatbot risks for children and teens.
- [Vals Public Sector Memo](https://www.vals.ai/blogs/vals-public-sector-memo): Introducing Vals Public Sector: independent AI evaluation for government, with expert-built benchmarks, frontier risk measurement and national security work.
- [Measuring Healthcare AI Where It Actually Works](https://www.vals.ai/blogs/measuring-healthcare-ai-where-it-actually-works): Compare healthcare AI on clinical SOAP notes and medical coding. Explore MedScribe and MedCode findings, including the gap between documentation and billing.
- [Behind the Scenes of Vibe Code Bench](https://www.vals.ai/blogs/behind-the-scenes-of-vibe-code-bench): How Vals AI built Vibe Code Bench to evaluate coding agents on web apps. Explore the benchmark design, testing approach and lessons from development.
- [News](https://www.vals.ai/news)
- [Media coverage](https://www.vals.ai/media)
- [Updates](https://www.vals.ai/updates)
- [Government](https://www.vals.ai/gov)

## Reports

- [Vals Legal AI Report](https://www.vals.ai/industry-reports/vlair-2-27-25): The first Vals Legal AI Report benchmarks CoCounsel, Vincent AI, Harvey Assistant, and Oliver across seven legal tasks against a lawyer baseline.
- [Alexi evaluation report](https://www.vals.ai/application-reports/alexi): Alexi scored 80% accuracy, 75% authoritativeness, and 70% appropriateness in VLAIR's 200-question legal research study, versus a lawyer baseline.
- [CoCounsel evaluation report](https://www.vals.ai/application-reports/cocounsel): CoCounsel's VLAIR scores across four legal tasks: Document Q&A, Chronology Generation, Document Summarization, and Data Extraction, vs a lawyer baseline.
- [Counsel Stack evaluation report](https://www.vals.ai/application-reports/counselstack): Counsel Stack scored 81% accuracy, 77% authoritativeness, and 71% appropriateness in VLAIR's 200-question legal research study, versus a lawyer baseline.
- [Harvey Assistant evaluation report](https://www.vals.ai/application-reports/harvey): Harvey Assistant's VLAIR scores across six legal tasks, including Document Q&A, Transcript Analysis, and Redlining, benchmarked against a lawyer baseline.
- [Midpage evaluation report](https://www.vals.ai/application-reports/midpage): Midpage scored 78% accuracy, 74% authoritativeness, and 70% appropriateness in VLAIR's 200-question legal research study, versus a lawyer baseline.
- [Oliver evaluation report](https://www.vals.ai/application-reports/oliver): Oliver's VLAIR scores across five legal tasks, including Document Q&A, Document Summarization, and EDGAR Research, compared with a lawyer baseline.
- [Vincent AI evaluation report](https://www.vals.ai/application-reports/vincent): Vincent AI's VLAIR scores across five legal tasks, including Document Q&A, Transcript Analysis, and Redlining, benchmarked against a lawyer baseline.

## Evaluation tools

- [Vals Smith](https://www.vals.ai/vals-smith)

## Machine-readable discovery

- [Full leaderboards](https://www.vals.ai/llms-full.txt): top models and scores for every current benchmark
- [Blog RSS feed](https://www.vals.ai/blogs/rss.xml)
- [Sitemap index](https://www.vals.ai/sitemap-index.xml)
- [Public URL sitemap](https://www.vals.ai/sitemap.xml)