The Public Standard for Real World AI Performance

Benchmark consisting of a weighted performance across finance and coding tasks. Showing the potential impact that LLMs can have on the economy.

Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLMs can have on the economy.

Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools.

Evaluating language models on a wide range of open source legal reasoning tasks.

A private benchmark evaluating understanding of long-context credit agreements

Evaluating agents on core financial analyst tasks

Evaluating reading and understanding tax certificates as images

A Vals-created set of questions and responses to tax questions

Can models support the medical billing process?

Can models support doctors with their administrative work?

Can models write math proofs that are formally verified?

Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

Multimodal Multi-task Benchmark

Student Assessment with Generative Evaluation

Can language models reimplement working programs in another language?

International Olympiad in Informatics

Our Implementation of the LiveCodeBench benchmark

Can language models rebuild programs from scratch?

Solving production software engineering tasks

State-of-the-art set of difficult terminal-based tasks

Can models build web applications from scratch?

Which model can make the most money playing poker?

Can AI help people navigate SNAP benefits?