The Public Standard for Real World AI Performance

Benchmark consisting of a weighted performance across finance, law and coding tasks. Showing the potential impact that LLM's can have on the economy.

Benchmark consisting of a weighted performance across finance, law, coding, and education tasks. Showing the potential impact that LLM's can have on the economy.

Private question-answer benchmark over Canadian court-cases.

Evaluating language models on a wide range of open source legal reasoning tasks.

A private benchmark evaluating understanding of long-context credit agreements

Evaluating agents on core financial analyst tasks

Evaluating reading and understanding tax certificates as images

A Vals-created set of questions and responses to tax questions

Can models support the medical billing process?

Can models support doctors with their administrative work?

Challenging national math exam given to top high-school students

Can models write math proofs that are formally verified?

Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

Multimodal Multi-task Benchmark

Student Assessment with Generative Evaluation

International Olympiad in Informatics

Our Implementation of the LiveCodeBench benchmark

Solving production software engineering tasks

State-of-the-art set of difficult terminal-based tasks

Can models build web applications from scratch?

Which model can make the most money playing poker?