Archived Benchmark

Since performance on this benchmark has saturated, we no longer run this benchmark on new model releases. Previous results are preserved here for posterity.

Academic

LiveCodeBench

Updated 9/1/2026

Our Implementation of the LiveCodeBench benchmark

As of September 1, 2026, Claude Fable 5.1 ranks first on LiveCodeBench with 90.52%, followed by Claude Fable 5 (89.78%) and Gemini 3.8 Flash (89.48%).

LiveCodeBenchLive competitive coding problems
ACCURACY

LiveCodeBench leaderboard

Rank Model Accuracy Cost In / Out Latency
1 Claude Fable 5.1 90.52% $10 / $50 59.74s
2 Claude Fable 5 89.78% $10 / $50 118.53s
3 Gemini 3.8 Flash 89.48% $1.5 / $7.5 24.07s
4 Claude Opus 5 89.03% $5 / $25 59.22s
5 Gemini 3.7 Flash 88.65% $1.5 / $7.5 14.53s
6 Gemini 3.1 Pro Preview (02/26) 88.48% $2 / $12 88.89s
7 Grok 4.6 88.22% $2 / $6 118.40s
8 Gemini 3.6 Flash 88.08% $1.5 / $7.5 31.88s
9 GPT 5.2 Codex 87.99% $1.75 / $14 2m48s
10 Qwen 3.8 Max 87.85% $2 / $6 5m38s
11 Claude Opus 4.8 87.82% $5 / $25 2m40s
12 Gemini 3.5 Flash 87.60% $1.5 / $9 64.37s
13 DeepSeek V4 Pro 0813 87.53% $1.32 / $3.96 4m33s
14 DeepSeek V4 87.48% $1.32 / $3.96 11m56s
15 Grok 4.5 87.35% $2 / $6 9m49s
16 GPT 5.3 Codex 87.31% $1.75 / $14 2m13s
17 DeepSeek V4 Flash 0731 87.26% $0.44 / $1.32 2m07s
18 Kimi K3 87.19% $3 / $15 3m20s
19 Qwen 3.7 Max 87.06% $2.5 / $7.5 3m12s
20 Kimi K2.6 86.77% $0.95 / $4 8m49s
21 GPT 5 Mini 86.61% $0.25 / $2 33.67s
22 GPT 5.1 86.49% $1.25 / $10 2m21s
23 Gemini 3 Pro (11/25) 86.41% $2 / $12 87.23s
24 Nemotron 3 Ultra 85.98% N/A 2m41s
25 Qwen 3.6 Plus 85.95% $0.5 / $3 7m56s
26 Inkling Small 85.93% $0.3 / $1.2 5m02s
27 GPT-5.6 Terra 85.93% $2 / $12 44.33s
28 GPT 5 85.91% $1.25 / $10 2m30s
29 Muse Spark 1.1 85.91% $1.25 / $4.25 113.25s
30 Gemini 3 Flash (12/25) 85.59% $0.5 / $3 91.93s
31 GPT 5.1 Codex 85.55% $1.25 / $10 3m54s
32 Inkling 85.52% $1 / $4.05 4m18s
33 GPT 5.2 85.36% $1.75 / $14 2m32s
34 Qwen 3.5 Plus 85.33% $0.4 / $2.4 4m07s
35 GPT 5.5 85.30% $5 / $30 2m47s
36 Claude Opus 4.7 85.07% $5 / $25 7m42s
37 GPT 5 Codex 84.72% $1.25 / $10 2m15s
38 Claude Opus 4.6 (Thinking) 84.68% $5 / $25 5m24s
39 Grok 4.3 84.49% $1.25 / $2.5 54m09s
40 Grok 4.20 (Reasoning) 84.27% $2 / $6 76.09s
41 GPT 5.4 (xhigh) 84.14% $2.5 / $15 3m29s
42 GPT 5.4 Nano 84.01% $0.2 / $1.25 60.09s
43 Qwen 3.8 27B 84.00% $0.5 / $3 2m29s
44 Ling 3.0 Flash 83.99% $0.075 / $0.22 60.40s
45 o3 83.91% $2 / $8 63.95s
46 Kimi K2.5 83.87% $0.6 / $3 3m20s
47 Claude Opus 4.5 (Thinking) 83.67% $5 / $25 2m44s
48 GPT 5.1 Codex Max 83.56% $1.25 / $10 66.20s
49 Qwen 3.5 Flash 83.28% $0.1 / $0.4 3m29s
50 Grok 4 83.25% $3 / $15 3m49s
51 GPT OSS 120B 83.23% $0.15 / $0.6 81.70s
52 GPT-5.6 Sol 82.60% $4 / $20 56.51s
53 Claude Sonnet 5 82.43% $2 / $10 77.02s
54 GLM 4.7 82.23% $0.6 / $2.2 6m33s
55 o4 Mini 82.21% $1.1 / $4.4 32.84s
56 MiniMax-M3 82.15% $0.6 / $2.4 6m07s
57 Claude Sonnet 4.6 82.09% $3 / $15 4m55s
58 Kimi K2.7 Code 82.05% $0.95 / $4 3m34s
59 GLM 5 81.87% $1 / $3.2 4m50s
60 MiniMax-M2.1 81.76% $0.3 / $1.2 4m06s
61 MiMo V2.5 81.51% $0.14 / $0.28 108.61s
62 GPT 5.4 Mini 81.47% $0.75 / $4.5 3m09s
63 GLM 5.1 81.38% $1 / $3.2 3m46s
64 MiMo V2.5 Pro 81.35% $0.435 / $0.87 4m31s
65 GLM 4.6 81.04% $0.6 / $2.2 3m56s
66 DeepSeek V3.2 (Thinking) 80.69% $0.56 / $1.68 4m14s
67 Grok 4.1 Fast (Reasoning) 80.64% $0.2 / $0.5 103.45s
68 GLM 5.3 80.53% $1.4 / $4.4 4m07s
69 GLM 5.3 Flash 80.51% $0.075 / $0.25 5m02s
70 GPT OSS 20B 80.39% $0.07 / $0.3 109.45s
71 Gemini 3.1 Flash Lite Preview 80.12% $0.25 / $1.5 30.75s
72 MiniMax-M2.7 79.93% $0.3 / $1.2 5m19s
73 MiniMax-M2.5 79.21% $0.3 / $1.2 5m12s
74 Gemini 2.5 Pro Preview 79.16% $1.25 / $10 2m45s
75 Gemini 3.5 Flash Lite 79.01% $0.3 / $2.5 21.08s
76 Grok 4 Fast (Reasoning) 78.97% $0.2 / $0.5 51.14s
77 Qwen 3 Max 78.22% $1.2 / $6 107.84s
78 Grok 3 Mini Reasoning 76.22% $0.3 / $0.5 3m34s
79 Gemini 2.5 Flash Preview (9/25) (Thinking) 76.21% $0.3 / $2.5 48.67s
80 Gemini 2.5 Flash Preview (9/25) (Nonthinking) 75.06% $0.3 / $2.5 46.78s
81 Claude Opus 4.5 (Nonthinking) 75.03% $5 / $25 13.60s
82 Magistral Medium 1.2 (09/2025) 74.86% $2 / $5 3m30s
83 Claude Sonnet 4.5 (Thinking) 73.00% $3 / $15 109.55s
84 Magistral Small 1.2 (09/2025) 72.13% $0.5 / $1.5 100.78s
85 o3 Mini 71.48% $1.1 / $4.4 53.80s
86 Gemini 2.5 Flash Lite (9/25) (Thinking) 71.39% $0.1 / $0.4 38.70s
87 Qwen 3 (235B) 70.62% $0.22 / $0.88 7m09s
88 Kimi K2 Instruct 70.45% $1 / $3 66.65s
89 DeepSeek R1 70.22% $1.35 / $5.4 86.07s
90 GPT 5 Nano 70.22% $0.05 / $0.4 2m16s
91 Claude Opus 4 (Thinking) 70.19% $15 / $75 93.54s
92 DeepSeek V3.2 (Nonthinking) 69.86% $0.56 / $1.68 39.34s
93 GLM 5.2 69.50% $1.4 / $4.4 8m52s
94 Laguna M.1 68.12% N/A 3m55s
95 Laguna XS.2 67.80% N/A 3m06s
96 Gemini 2.5 Flash Lite (9/25) (Nonthinking) 67.67% $0.1 / $0.4 17.44s
97 GLM 4.5 67.45% $0.6 / $2.2 6m31s
98 Qwen 3 Max Preview 66.91% $1.2 / $6 96.25s
99 Claude Opus 4.1 (Thinking) 66.46% $15 / $75 92.17s
100 Grok 3 Mini Reasoning 66.27% $0.3 / $0.5 30.71s
101 DeepSeek V3 (03/24/2025) 65.48% $0.9 / $0.9 26.93s
102 Claude Opus 4.1 (Nonthinking) 64.56% $15 / $75 32.51s
103 Kimi K2 Thinking 63.15% $0.6 / $2.5 19m11s
104 Claude Opus 4 (Nonthinking) 62.63% $15 / $75 14.78s
105 Claude Sonnet 4 (Thinking) 62.39% $3 / $15 42.61s
106 Grok Code Fast 61.97% $0.2 / $1.5 25.84s
107 Claude 3.7 Sonnet (Thinking) 60.44% $3 / $15 96.47s
108 Claude Sonnet 4 (Nonthinking) 59.67% $3 / $15 14.06s
109 Llama 3.3 Nemotron Super (Thinking) 58.37% N/A 2m32s
110 GPT 4.1 Mini 58.16% $0.4 / $1.6 27.59s
111 Gemini 2.5 Flash Preview 4/17 (Nonthinking) 56.94% $0.3 / $2.5 16.41s
112 Claude 3.7 Sonnet (Nonthinking) 56.66% $3 / $15 16.23s
113 Mistral Large 3 55.34% $0.5 / $1.5 28.89s
114 GPT 4.1 54.67% $2 / $8 30.57s
115 Grok 3 52.90% $3 / $15 5.52s
116 Devstral 2 51.84% $0.4 / $2 87.57s
117 o1 50.26% $15 / $60 92.08s
118 Claude 3.5 Sonnet Latest 49.63% $3 / $15 11.04s
119 Llama 4 Maverick 47.25% $0.22 / $0.88 9.44s
120 Gemini 2.5 Flash Preview 4/17 (Thinking) 46.87% $0.3 / $2.5 2m05s
121 Grok 4 Fast (Non-Reasoning) 46.09% $0.2 / $0.5 8.96s
122 Mistral Medium 3.1 (05/2025) 44.84% $0.4 / $2 12.48s
123 Gemini 2.0 Flash (001) 43.61% $0.1 / $0.4 3.36s
124 GPT 4o (2024-11-20) 43.44% $2.5 / $10 4.35s
125 Devstral Small 2 43.18% $0.1 / $0.3 18.82s
126 GPT 4.1 Nano 42.72% $0.1 / $0.4 11.80s
127 Grok 4.1 Fast Non-Reasoning 42.62% $0.2 / $0.5 11.96s
128 Claude 3.5 Haiku Latest 41.92% $0.8 / $4 10.58s
129 Gemini 1.5 Pro (002) 41.72% $1.25 / $5 3.95s
130 Claude Haiku 4.5 (Thinking) 41.17% $1 / $5 81.56s
131 Grok 2 38.68% $2 / $10 2.38s
132 Llama 4 Scout 38.54% $0.18 / $0.59 16.76s
133 Mistral Large (11/2024) 37.09% $2 / $6 5.35s
134 Gemini 1.5 Flash (002) 36.91% $0.075 / $0.3 2.51s
135 Llama 3.3 Instruct Turbo (70B) 36.34% $0.88 / $0.88 2.94s
136 Llama 3.3 Nemotron Super (Nonthinking) 36.31% N/A 44.23s
137 Command A 35.07% $2.5 / $10 11.75s
138 Mistral Small 3.1 (03/2025) 31.82% $0.075 / $0.3 4.01s
139 GPT 4o Mini 26.42% $0.15 / $0.6 7.36s
140 Jamba 1.6 Large 22.32% $2 / $8 4.84s
141 Command R+ 18.24% $2.5 / $10 5.46s
142 Mistral Small (02/2024) 15.78% $0.2 / $0.6 5.04s
143 Jamba 1.6 Mini 9.92% $0.2 / $0.4 1.14s

Dataset and Context

LiveCodeBench is a programming benchmark designed to assess the capabilities of LLMs on competitive programming problems. LiveCodeBench continuously collects new coding problems from competitive programming platforms, providing a dynamic evaluation framework that evolves over time.

Problem Sources and Scale

This benchmark evaluates models on coding problems collected from three major competitive programming platforms:

  • LeetCode: Industry-standard coding interview problems
  • AtCoder: Japanese competitive programming platform known for algorithmic challenges
  • Codeforces: International platform hosting regular programming contests

The newest version (v6) of the benchmark includes over 1000 high-quality coding problems collected between May 2023 and 2025, with problems categorized into three difficulty levels: easy, medium, and hard. Each problem consists of a natural language problem statement, example input-output pairs, and hidden test cases for evaluation.

Evaluation Approach

The “Code Generation” task in LiveCodeBench evaluates models on code generation tasks where:

  1. Models receive a problem statement with natural language description and example tests
  2. They must generate a syntactically correct Python solution
  3. Solutions are evaluated against hidden test cases for functional correctness

This approach tests not just syntax generation but also:

  • Problem comprehension and decomposition
  • Algorithm design and implementation
  • Edge case handling
  • Code efficiency considerations

Why LiveCodeBench Matters

While benchmarks like HumanEval have become saturated (with many models achieving near-perfect scores), LiveCodeBench remains challenging due to:

  • Higher problem complexity from real competitive programming
  • Continuous addition of new problems mitigating overfitting and data contamination
  • Diverse problem types requiring various algorithmic approaches
  • Strict functional correctness evaluation with hidden test cases

Results

After accounting for price and latency, OpenAI’s GPT 5 Mini stands out as a strong option, combining high performance (86.61%) with low cost. Claude Fable 5 leads overall at 89.78%, followed by Claude Opus 5 (89.03%), Google’s Gemini 3.1 Pro Preview (02/26) (88.48%), Gemini 3.6 Flash (88.08%), and OpenAI’s GPT 5.2 Codex (87.99%).


We see significant differences in model performance as a function of split. Existing models are near-perfect on the easy split, perform well on the medium split, and struggle on the hard split.

We also find that performance on the medium and hard splits is correlated. In other words, models that are good at the medium problems are also good at the hard problems. This may be unsurprising, but is also less obvious for easy problems, since some models are good at the easy problems but still struggle on the medium and hard problems.