Archived Benchmark

Since performance on this benchmark has saturated, we no longer run this benchmark on new model releases. Previous results are preserved here for posterity.

Academic

IOI

Updated 8/9/2026

International Olympiad in Informatics

As of August 9, 2026, Claude Opus 5 ranks first on IOI v1 with 91.67%, followed by GPT-5.6 Sol (86.67%) and Qwen 3.8 Max (73.00%).

IOI v1Informatics olympiad programming problems
IOI v1•Aug 09, 2026
ACCURACY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

IOI leaderboard

Rank Model Accuracy Cost In / Out Tokens / Test Duration / Test
1 Claude Opus 5 91.67% $5 / $25 - 28m13s
2 GPT-5.6 Sol 86.67% $4 / $20 1.41m 21m51s
3 Qwen 3.8 Max 73.00% $2 / $6 - 1h06m
4 GPT-5.6 Luna 72.92% $0.2 / $1.2 - 27m51s
5 Claude Fable 5 72.25% $10 / $50 - 55m19s
6 GPT 5.4 (xhigh) 67.83% $2.5 / $15 3.24m 44m21s
7 GPT-5.6 Terra 65.25% $2 / $12 - 22m11s
8 GPT 5.2 54.83% $1.75 / $14 2.29m 1h52m
9 Muse Spark 1.2 49.50% $1.25 / $4.25 15.64m 1h20m
10 Claude Opus 4.7 47.08% $5 / $25 4.19m 51m22s
11 Qwen 3.7 Max 46.75% $2.5 / $7.5 3.18m 1h13m
12 GPT 5.3 Codex 43.83% $1.75 / $14 2.23m 59m34s
13 Gemini 3 Flash (12/25) 39.08% $0.5 / $3 1.36m 30m47s
14 Gemini 3 Pro (11/25) 38.83% $2 / $12 114.41k 15m00s
15 DeepSeek V4 35.83% $1.32 / $3.96 6.95m 1h30m
16 Grok 4.20 (Reasoning) 30.17% $2 / $6 685.82k 14m57s
17 Gemini 3.5 Flash Lite 26.17% $0.3 / $2.5 4.10m 14m42s
18 Grok 4 26.17% $3 / $15 716.20k 1h11m
19 Claude Opus 4.5 (Nonthinking) 23.58% $5 / $25 - 2h08m
20 GLM 5 22.00% $1 / $3.2 4.96m 1h53m
21 GPT 5.1 21.50% $1.25 / $10 197.56k 29m10s
22 GPT 5.1 Codex Max 21.42% $1.25 / $10 266.88k 15m57s
23 Claude Opus 4.5 (Thinking) 20.25% $5 / $25 157.24k 2h17m
24 GPT 5 20.00% $1.25 / $10 404.28k 43m28s
25 Claude Sonnet 4.5 (Thinking) 18.33% $3 / $15 846.62k 55m15s
26 Kimi K2.5 17.67% $0.6 / $3 2.36m 1h04m
27 Gemini 2.5 Pro 17.08% $1.25 / $10 306.21k 49m25s
28 Qwen 3 Max 15.67% $1.2 / $6 553.59k 34m05s
29 Grok 4.3 15.33% $1.25 / $2.5 628.42k 10m29s
30 GPT 5.4 Nano 15.25% $0.2 / $1.25 964.13k 19m30s
31 DeepSeek V3.2 (Nonthinking) 14.42% $0.56 / $1.68 33.76k 39m45s
32 Qwen 3 Max Thinking 13.75% $1.2 / $6 603.62k 49m13s
33 Claude Opus 4.1 (Nonthinking) 12.52% $15 / $75 1.03m 1h42m
34 Grok 4 Fast (Reasoning) 11.50% $0.2 / $0.5 2.75m 23m28s
35 DeepSeek V3.2 (Thinking) 10.67% $0.56 / $1.68 68.73k 25m59s
36 GPT 5 Codex 9.75% $1.25 / $10 1.52m 25m43s
37 Qwen 3 Max Preview 7.75% $1.2 / $6 1.31m 56m12s
38 Grok 4.1 Fast Non-Reasoning 7.67% $0.2 / $0.5 301.67k 2h08m
39 GLM 4.7 7.58% $0.6 / $2.2 1.60m 1h29m
40 GPT 5 Mini 6.75% $0.25 / $2 80.73k 6m30s
41 MiniMax-M2.5 6.67% $0.3 / $1.2 4.42m 6h21m
42 Claude Sonnet 4 (Nonthinking) 6.50% $3 / $15 703.15k 54m13s
43 GPT 5.4 Mini 6.42% $0.75 / $4.5 1.40m 31m37s
44 Claude Haiku 4.5 (Thinking) 6.17% $1 / $5 1.86m 2h04m
45 MiniMax-M2.7 4.92% $0.3 / $1.2 3.53m 54m03s
46 o4 Mini 4.83% $1.1 / $4.4 68.83k 8m00s
47 Claude Sonnet 4 (Thinking) 4.58% $3 / $15 377.94k 36m56s
48 GLM 4.6 4.33% $0.6 / $2.2 2.77m 1h58m
49 Grok Code Fast 4.33% $0.2 / $1.5 1.84m 18m04s
50 Mistral Large 3 4.00% $0.5 / $1.5 5.87m 1h53m
51 Grok 4 Fast (Non-Reasoning) 3.83% $0.2 / $0.5 357.07k 9m38s
52 GPT 5.1 Codex 3.67% $1.25 / $10 56.43k 37m07s
53 Grok 4.1 Fast (Reasoning) 3.08% $0.2 / $0.5 134.56k 25m33s
54 GLM 4.5 2.92% $0.6 / $2.2 1.37m 38m56s
55 Gemini 2.5 Flash (7/17) (Nonthinking) 2.61% $0.3 / $2.5 66.35k 5m50s
56 Devstral Small 2 2.50% $0.1 / $0.3 4.88m 2h04m
57 MiniMax-M2.1 2.33% $0.3 / $1.2 1.69m 1h27m
58 DeepSeek V3 (03/24/2025) 1.67% $0.9 / $0.9 673.87k 2m51s
59 Kimi K2 Instruct 1.25% $1 / $3 2.29m 25m51s
60 Magistral Medium 1.2 (09/2025) 0.67% $2 / $5 3.05m 36m52s
61 Devstral 2 0.67% $0.4 / $2 4.13m 2h14m
62 Qwen 3 (235B) 0.00% $0.22 / $0.88 406.36k 5m53s

Why IOI?

Recently, top LLM labs like OpenAI and Google reported that their models achieved gold medals on the International Mathematical Olympiad (IMO). However, advanced models are starting to saturate IMO, meaning it may no longer effectively differentiate between the capabilities of top-performing models. Reports also suggest the evaluation process faced coordination challenges, with AI companies seeking expedited validation mid-competition that may not reflect standard IMO assessment procedures.

The International Olympiad in Informatics (IOI) offers several advantages as an LLM benchmark. Unlike the IMO, the IOI is not yet saturated, providing clear differentiation between model capabilities. The competition features standardized and automated grading, ensuring objective evaluation without subjective scoring. Additionally, the IOI has real-world relevance as it tests C++ programming skills that are directly applicable to software development.

Benchmark Design

We designed our benchmark to imitate competition conditions as closely as possible.

Agent Harness

We adapted our open-source agent harness from our Finance Agent Benchmark by providing it with access to the following tools:

  1. a c++ (v20) execution environment, which executes arbitrary code
  2. a submission tool used for grading, which executes submitted code and returns a score out of 100 total possible points

These tools, particularly the submission tool, were based on the testing environment available to human contestants.

Our implementation is open source in the IOI agent GitHub repository.

Scoring

We further designed our submission tool to match the grading process during the olympiad. Agents get up to 50 submissions, each of which are graded on a variety of subtasks. A competitor receives credit for a given subtask if any submission passes all tests on a given subtask. In particular, this means the final score can be higher than any individual submission, since credit for separate subtasks are combined by the grading system.

Results

Claude Opus 5 leads with 91.67% accuracy, followed by GPT-5.6 Sol at 86.67% and Qwen 3.8 Max at 73.00%. GPT-5.6 Luna (72.92%) and Claude Fable 5 (72.25%) complete the top five. Performance remains uneven across years and models, with a steep drop after the top two results.

We chose to evaluate on both 2024 and 2025 to check for data contamination, which we suspect to explain the performance decrease between those years on LiveCodeBench. Across the current model cohort, average performance is essentially flat between the two years, with 2025 slightly higher.