Why IOI?
Recently, top LLM labs like OpenAI and Google reported that their models achieved gold medals on the International Mathematical Olympiad (IMO). However, advanced models are starting to saturate IMO, meaning it may no longer effectively differentiate between the capabilities of top-performing models. Reports also suggest the evaluation process faced coordination challenges, with AI companies seeking expedited validation mid-competition that may not reflect standard IMO assessment procedures.
The International Olympiad in Informatics (IOI) offers several advantages as an LLM benchmark. The competition features standardized and automated grading, ensuring objective evaluation without subjective scoring. Additionally, the IOI has real-world relevance as it tests C++ programming skills that are directly applicable to software development.
Benchmark Design
We designed our benchmark to imitate competition conditions as closely as possible.
Agent Harness
Each model runs inside OpenCode, a general-purpose coding agent, in an isolated sandbox with a c++ (v20) toolchain. Its workspace holds exactly what the contest hands a contestant: the problem statement, the task header and solution stub, the sample grader or manager, the compile and run scripts, and the sample input and output from the statement. The official test data, the subtask test lists, and the grading script are withheld until the attempt is over.
The agent reads, writes, compiles and tests files freely in its workspace, and writes its answer to /workspace/solution.cpp. Like a contestant, it has no access to the public internet: the sandbox reaches only our model gateway, so published editorials and reference solutions are out of reach.
Unlike a contestant, the agent has no submission tool and therefore receives no score feedback during the attempt: it can run the samples and whatever tests it writes for itself, and must decide on its own when its solution is good enough.
Some models spend their entire per-response output budget reasoning about a problem before writing any code. When that happens, the harness continues the cut-off turn on the next step rather than failing the problem: it replays the provider’s own reasoning state where the API returns one, and otherwise restarts the turn with an instruction to work in smaller steps. The same rule applies to every model on this board, and the cost column includes the reasoning spent this way.
Results from our earlier harness, which gave the agent an interactive grading tool, are preserved on IOI v1. The two harnesses are not comparable, so their results are reported on separate boards.
Scoring
Grading matches the olympiad: after the attempt ends, the solution left in the workspace is copied into a fresh grading directory alongside pristine copies of the official test data and grader, compiled, run against every official test, and scored per subtask out of 100 points. Every grading file is verified unchanged after the run. A solution that fails to compile, or that the agent never wrote, scores zero. A model’s score for a year is its mean score across that year’s six problems, and its overall score is the mean of its three yearly scores.
Results
Gemini 4 Argon (100.00%), GPT-6 Astra (100.00%), and GPT-6.1 Sol (96.89%) lead the board. Claude Sonnet 5 is expensive: on thirteen of its eighteen problems it spent its entire 128k-token output budget reasoning before writing any code, and the harness had to continue the cut-off turn.
We evaluate three years to check for data contamination. The 2026 problems were released only after the models in this cohort were trained, and they are the hardest set for many of them. Gemini 4 Argon scores 100.00% on 2026, so the newest set is not out of reach.