Archived Benchmark

Since performance on this benchmark has saturated, we no longer run this benchmark on new model releases. Previous results are preserved here for posterity.

Academic

LiveCodeBench

Updated 9/1/2026

Our Implementation of the LiveCodeBench benchmark

As of September 1, 2026, Claude Fable 5.1 ranks first on LiveCodeBench with 90.52%, followed by Claude Fable 5 (89.78%) and Gemini 3.8 Flash (89.48%).

LiveCodeBenchLive competitive coding problems
LiveCodeBench•Sep 01, 2026
ACCURACY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

LiveCodeBench leaderboard

Rank Model Accuracy Cost In / Out Tokens / Test Latency
1 Claude Fable 5.1 90.52% $10 / $50 5.80k 59.74s
2 Claude Fable 5 89.78% $10 / $50 9.28k 118.53s
3 Gemini 3.8 Flash 89.48% $0.75 / $3.75 9.98k 24.07s
4 Claude Opus 5 89.03% $5 / $25 5.95k 59.22s
5 Gemini 3.7 Flash 88.65% $1.5 / $7.5 5.85k 14.53s
6 Gemini 3.1 Pro Preview (02/26) 88.48% $2 / $12 8.93k 88.89s
7 Grok 4.6 88.22% $2 / $6 7.09k 118.40s
8 Gemini 3.6 Flash 88.08% $0.75 / $3.75 8.00k 31.88s
9 GPT 5.2 Codex 87.99% $1.75 / $14 17.57k 2m48s
10 Qwen 3.8 Max 87.85% $2 / $6 15.67k 5m38s
11 Claude Opus 4.8 87.82% $5 / $25 13.71k 2m40s
12 Gemini 3.5 Flash 87.60% $1.5 / $9 10.05k 64.37s
13 DeepSeek V4 Pro 0813 87.53% $1.32 / $3.96 - 4m33s
14 DeepSeek V4 87.48% $1.32 / $3.96 28.13k 11m56s
15 Grok 4.5 87.35% $2 / $6 7.93k 9m49s
16 GPT 5.3 Codex 87.31% $1.75 / $14 7.37k 2m13s
17 DeepSeek V4 Flash 0731 87.26% $0.3 / $1.2 - 2m07s
18 Kimi K3 87.19% $3 / $15 7.54k 3m20s
19 Qwen 3.7 Max 87.06% $2.5 / $7.5 11.97k 3m12s
20 Kimi K2.6 86.77% $0.95 / $4 17.26k 8m49s
21 GPT 5 Mini 86.61% $0.25 / $2 6.26k 33.67s
22 GPT 5.1 86.49% $1.25 / $10 13.07k 2m21s
23 Gemini 3 Pro (11/25) 86.41% $2 / $12 11.12k 87.23s
24 Nemotron 3 Ultra 85.98% - 15.40k 2m41s
25 Qwen 3.6 Plus 85.95% $0.5 / $3 26.39k 7m56s
26 Inkling Small 85.93% $0.3 / $1.2 17.87k 5m02s
27 GPT-5.6 Terra 85.93% $2 / $12 3.16k 44.33s
28 GPT 5 85.91% $1.25 / $10 7.37k 2m30s
29 Muse Spark 1.1 85.91% $1.25 / $4.25 11.58k 113.25s
30 Gemini 3 Flash (12/25) 85.59% $0.5 / $3 17.37k 91.93s
31 GPT 5.1 Codex 85.55% $1.25 / $10 14.25k 3m54s
32 Inkling 85.52% $1 / $4.05 16.75k 4m18s
33 GPT 5.2 85.36% $1.75 / $14 19.63k 2m32s
34 Qwen 3.5 Plus 85.33% $0.4 / $2.4 20.45k 4m07s
35 GPT 5.5 85.30% $5 / $30 4.64k 2m47s
36 Claude Opus 4.7 85.07% $5 / $25 18.29k 7m42s
37 GPT 5 Codex 84.72% $1.25 / $10 19.67k 2m15s
38 Claude Opus 4.6 (Thinking) 84.68% $5 / $25 23.41k 5m24s
39 Grok 4.3 84.49% $1.25 / $2.5 8.47k 54m09s
40 Grok 4.20 (Reasoning) 84.27% $2 / $6 9.23k 76.09s
41 GPT 5.4 (xhigh) 84.14% $2.5 / $15 11.89k 3m29s
42 GPT 5.4 Nano 84.01% $0.2 / $1.25 6.90k 60.09s
43 Qwen 3.8 27B 84.00% $0.5 / $3 15.54k 2m29s
44 Ling 3.0 Flash 83.99% $0.075 / $0.22 14.14k 60.40s
45 o3 83.91% $2 / $8 4.54k 63.95s
46 Kimi K2.5 83.87% $0.6 / $3 13.46k 3m20s
47 Claude Opus 4.5 (Thinking) 83.67% $5 / $25 13.47k 2m44s
48 GPT 5.1 Codex Max 83.56% $1.25 / $10 851 66.20s
49 Qwen 3.5 Flash 83.28% $0.1 / $0.4 24.31k 3m29s
50 Grok 4 83.25% $3 / $15 10.85k 3m49s
51 GPT OSS 120B 83.23% $0.15 / $0.6 14.97k 81.70s
52 GPT-5.6 Sol 82.60% $4 / $20 3.81k 56.51s
53 Claude Sonnet 5 82.43% $2 / $10 6.91k 77.02s
54 GLM 4.7 82.23% $0.6 / $2.2 16.80k 6m33s
55 o4 Mini 82.21% $1.1 / $4.4 3.53k 32.84s
56 MiniMax-M3 82.15% $0.3 / $1.2 16.18k 6m07s
57 Claude Sonnet 4.6 82.09% $3 / $15 25.21k 4m55s
58 Kimi K2.7 Code 82.05% $0.95 / $4 7.27k 3m34s
59 GLM 5 81.87% $1 / $3.2 14.22k 4m50s
60 MiniMax-M2.1 81.76% $0.3 / $1.2 15.80k 4m06s
61 MiMo V2.5 81.51% $0.14 / $0.28 8.30k 108.61s
62 GPT 5.4 Mini 81.47% $0.75 / $4.5 17.89k 3m09s
63 GLM 5.1 81.38% $1 / $3.2 14.76k 3m46s
64 MiMo V2.5 Pro 81.35% $0.435 / $0.87 8.50k 4m31s
65 GLM 4.6 81.04% $0.6 / $2.2 13.16k 3m56s
66 DeepSeek V3.2 (Thinking) 80.69% $0.56 / $1.68 8.73k 4m14s
67 Grok 4.1 Fast (Reasoning) 80.64% $0.2 / $0.5 8.26k 103.45s
68 GLM 5.3 80.53% $1.4 / $4.4 16.16k 4m07s
69 GLM 5.3 Flash 80.51% $0.075 / $0.25 28.62k 5m02s
70 GPT OSS 20B 80.39% $0.07 / $0.3 23.62k 109.45s
71 Gemini 3.1 Flash Lite Preview 80.12% $0.25 / $1.5 11.33k 30.75s
72 MiniMax-M2.7 79.93% $0.3 / $1.2 18.25k 5m19s
73 MiniMax-M2.5 79.21% $0.3 / $1.2 14.25k 5m12s
74 Gemini 2.5 Pro Preview 79.16% $1.25 / $10 2.54k 2m45s
75 Gemini 3.5 Flash Lite 79.01% $0.3 / $2.5 8.24k 21.08s
76 Grok 4 Fast (Reasoning) 78.97% $0.2 / $0.5 9.41k 51.14s
77 Qwen 3 Max 78.22% $1.2 / $6 4.01k 107.84s
78 Grok 3 Mini Reasoning 76.22% $0.3 / $0.5 16.34k 3m34s
79 Gemini 2.5 Flash Preview (9/25) (Thinking) 76.21% $0.3 / $2.5 13.31k 48.67s
80 Gemini 2.5 Flash Preview (9/25) (Nonthinking) 75.06% $0.3 / $2.5 13.19k 46.78s
81 Claude Opus 4.5 (Nonthinking) 75.03% $5 / $25 1.53k 13.60s
82 Magistral Medium 1.2 (09/2025) 74.86% $2 / $5 10.83k 3m30s
83 Claude Sonnet 4.5 (Thinking) 73.00% $3 / $15 9.50k 109.55s
84 Magistral Small 1.2 (09/2025) 72.13% $0.5 / $1.5 10.63k 100.78s
85 o3 Mini 71.48% $1.1 / $4.4 7.95k 53.80s
86 Gemini 2.5 Flash Lite (9/25) (Thinking) 71.39% $0.1 / $0.4 13.53k 38.70s
87 Qwen 3 (235B) 70.62% $0.22 / $0.88 15.95k 7m09s
88 Kimi K2 Instruct 70.45% $1 / $3 2.67k 66.65s
89 DeepSeek R1 70.22% $1.35 / $5.4 7.97k 86.07s
90 GPT 5 Nano 70.22% $0.05 / $0.4 13.16k 2m16s
91 Claude Opus 4 (Thinking) 70.19% $15 / $75 5.82k 93.54s
92 DeepSeek V3.2 (Nonthinking) 69.86% $0.56 / $1.68 1.82k 39.34s
93 GLM 5.2 69.50% $1.4 / $4.4 29.84k 8m52s
94 Laguna M.1 68.12% - 26.67k 3m55s
95 Laguna XS.2 67.80% - 22.14k 3m06s
96 Gemini 2.5 Flash Lite (9/25) (Nonthinking) 67.67% $0.1 / $0.4 5.46k 17.44s
97 GLM 4.5 67.45% $0.6 / $2.2 8.72k 6m31s
98 Qwen 3 Max Preview 66.91% $1.2 / $6 2.73k 96.25s
99 Claude Opus 4.1 (Thinking) 66.46% $15 / $75 5.33k 92.17s
100 Grok 3 Mini Reasoning 66.27% $0.3 / $0.5 4.53k 30.71s
101 DeepSeek V3 (03/24/2025) 65.48% $0.9 / $0.9 1.97k 26.93s
102 Claude Opus 4.1 (Nonthinking) 64.56% $15 / $75 1.24k 32.51s
103 Kimi K2 Thinking 63.15% $0.6 / $2.5 14.80k 19m11s
104 Claude Opus 4 (Nonthinking) 62.63% $15 / $75 1.24k 14.78s
105 Claude Sonnet 4 (Thinking) 62.39% $3 / $15 3.34k 42.61s
106 Grok Code Fast 61.97% $0.2 / $1.5 6.33k 25.84s
107 Claude 3.7 Sonnet (Thinking) 60.44% $3 / $15 8.83k 96.47s
108 Claude Sonnet 4 (Nonthinking) 59.67% $3 / $15 1.42k 14.06s
109 Llama 3.3 Nemotron Super (Thinking) 58.37% - 9.15k 2m32s
110 GPT 4.1 Mini 58.16% $0.4 / $1.6 1.16k 27.59s
111 Gemini 2.5 Flash Preview 4/17 (Nonthinking) 56.94% $0.3 / $2.5 5.22k 16.41s
112 Claude 3.7 Sonnet (Nonthinking) 56.66% $3 / $15 1.87k 16.23s
113 Mistral Large 3 55.34% $0.5 / $1.5 1.88k 28.89s
114 GPT 4.1 54.67% $2 / $8 1.59k 30.57s
115 Grok 3 52.90% $3 / $15 1.07k 5.52s
116 Devstral 2 51.84% $0.4 / $2 29.01k 87.57s
117 o1 50.26% $15 / $60 5.89k 92.08s
118 Claude 3.5 Sonnet Latest 49.63% $3 / $15 1.27k 11.04s
119 Llama 4 Maverick 47.25% $0.22 / $0.88 1.37k 9.44s
120 Gemini 2.5 Flash Preview 4/17 (Thinking) 46.87% $0.3 / $2.5 1.33k 2m05s
121 Grok 4 Fast (Non-Reasoning) 46.09% $0.2 / $0.5 2.43k 8.96s
122 Mistral Medium 3.1 (05/2025) 44.84% $0.4 / $2 1.28k 12.48s
123 Gemini 2.0 Flash (001) 43.61% $0.1 / $0.4 1.22k 3.36s
124 GPT 4o (2024-11-20) 43.44% $2.5 / $10 825 4.35s
125 Devstral Small 2 43.18% $0.1 / $0.3 1.57k 18.82s
126 GPT 4.1 Nano 42.72% $0.1 / $0.4 1.17k 11.80s
127 Grok 4.1 Fast Non-Reasoning 42.62% $0.2 / $0.5 1.36k 11.96s
128 Claude 3.5 Haiku Latest 41.92% $0.8 / $4 1.26k 10.58s
129 Gemini 1.5 Pro (002) 41.72% $1.25 / $5 849 3.95s
130 Claude Haiku 4.5 (Thinking) 41.17% $1 / $5 12.47k 81.56s
131 Grok 2 38.68% $2 / $10 762 2.38s
132 Llama 4 Scout 38.54% $0.18 / $0.59 1.44k 16.76s
133 Mistral Large (11/2024) 37.09% $2 / $6 989 5.35s
134 Gemini 1.5 Flash (002) 36.91% $0.075 / $0.3 945 2.51s
135 Llama 3.3 Instruct Turbo (70B) 36.34% $0.88 / $0.88 845 2.94s
136 Llama 3.3 Nemotron Super (Nonthinking) 36.31% - 844 44.23s
137 Command A 35.07% $2.5 / $10 1.13k 11.75s
138 Mistral Small 3.1 (03/2025) 31.82% $0.075 / $0.3 932 4.01s
139 GPT 4o Mini 26.42% $0.15 / $0.6 894 7.36s
140 Jamba 1.6 Large 22.32% $2 / $8 895 4.84s
141 Command R+ 18.24% $2.5 / $10 796 5.46s
142 Mistral Small (02/2024) 15.78% $0.2 / $0.6 1.08k 5.04s
143 Jamba 1.6 Mini 9.92% $0.2 / $0.4 824 1.14s

Dataset and Context

LiveCodeBench is a programming benchmark designed to assess the capabilities of LLMs on competitive programming problems. LiveCodeBench continuously collects new coding problems from competitive programming platforms, providing a dynamic evaluation framework that evolves over time.

Problem Sources and Scale

This benchmark evaluates models on coding problems collected from three major competitive programming platforms:

  • LeetCode: Industry-standard coding interview problems
  • AtCoder: Japanese competitive programming platform known for algorithmic challenges
  • Codeforces: International platform hosting regular programming contests

The newest version (v6) of the benchmark includes over 1000 high-quality coding problems collected between May 2023 and 2025, with problems categorized into three difficulty levels: easy, medium, and hard. Each problem consists of a natural language problem statement, example input-output pairs, and hidden test cases for evaluation.

Evaluation Approach

The “Code Generation” task in LiveCodeBench evaluates models on code generation tasks where:

  1. Models receive a problem statement with natural language description and example tests
  2. They must generate a syntactically correct Python solution
  3. Solutions are evaluated against hidden test cases for functional correctness

This approach tests not just syntax generation but also:

  • Problem comprehension and decomposition
  • Algorithm design and implementation
  • Edge case handling
  • Code efficiency considerations

Why LiveCodeBench Matters

While benchmarks like HumanEval have become saturated (with many models achieving near-perfect scores), LiveCodeBench remains challenging due to:

  • Higher problem complexity from real competitive programming
  • Continuous addition of new problems mitigating overfitting and data contamination
  • Diverse problem types requiring various algorithmic approaches
  • Strict functional correctness evaluation with hidden test cases

Results

After accounting for price and latency, OpenAI’s GPT 5 Mini stands out as a strong option, combining high performance (86.61%) with low cost. Claude Fable 5 leads overall at 89.78%, followed by Claude Opus 5 (89.03%), Google’s Gemini 3.1 Pro Preview (02/26) (88.48%), Gemini 3.6 Flash (88.08%), and OpenAI’s GPT 5.2 Codex (87.99%).


We see significant differences in model performance as a function of split. Existing models are near-perfect on the easy split, perform well on the medium split, and struggle on the hard split.

We also find that performance on the medium and hard splits is correlated. In other words, models that are good at the medium problems are also good at the hard problems. This may be unsurprising, but is also less obvious for easy problems, since some models are good at the easy problems but still struggle on the medium and hard problems.