Claude Sonnet 5.5 vs GPT-6.1 Sol: Benchmark Comparison

Claude Sonnet 5.5 has the higher score on 21 of 28 shared benchmarks; GPT-6.1 Sol leads on 6.

The largest observed score gap is 20.61 pts on SRE Bench , where GPT-6.1 Sol leads.

Reported ±1 standard-error ranges overlap on 11 of 28 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.

Shared benchmark results

Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.

Benchmark Claude Sonnet 5.5 GPT-6.1 Sol Gap Reported uncertainty
Vals Index 70.85% ±0.93 63.94% ±1.07 6.91 pts Reported ±1 SE ranges do not overlap
Vals Finance Index 64.53% ±1.48 55.81% ±1.50 8.72 pts Reported ±1 SE ranges do not overlap
Vals Legal Index 68.02% ±1.81 68.72% ±1.77 0.70 pts Reported ±1 SE ranges overlap
Vals Science Index 59.12% ±2.31 59.63% ±2.31 0.51 pts Reported ±1 SE ranges overlap
HLAB-Vals-1.1 10.42% ±2.57 6.25% ±1.83 4.17 pts Reported ±1 SE ranges overlap
Immigration 68.96% ±1.54 67.81% ±1.53 1.16 pts Reported ±1 SE ranges overlap
Legal Research Bench 48.08% ±3.47 38.46% ±3.38 9.61 pts Reported ±1 SE ranges do not overlap
ClaimBench 100.00% ±0.00 100.00% ±0.00 0.00 pts Reported ±1 SE ranges overlap
EMB v1.1 75.90% ±2.47 71.09% ±2.75 4.81 pts Reported ±1 SE ranges overlap
Finance Agent v2.1 67.28% ±2.02 61.82% ±2.11 5.46 pts Reported ±1 SE ranges do not overlap
Tax Agent Bench 41.96% ±3.56 21.75% ±2.98 20.21 pts Reported ±1 SE ranges do not overlap
MedCode 52.92% ±2.12 48.84% ±2.12 4.08 pts Reported ±1 SE ranges overlap
MedScribe 91.10% ±1.96 86.45% ±1.90 4.65 pts Reported ±1 SE ranges do not overlap
ProofBench v1.1 100.00% ±0.00 99.00% ±1.00 1.00 pts Reported ±1 SE ranges overlap
BioMysteryBench 81.11% ±0.64 79.63% ±0.74 1.48 pts Reported ±1 SE ranges do not overlap
MysteryMechanism 49.10% ±3.36 46.40% ±3.35 2.70 pts Reported ±1 SE ranges overlap
Terminal-Bench-Science 47.14% ±6.01 52.86% ±6.01 5.71 pts Reported ±1 SE ranges overlap
SAGE 88.03% ±1.62 90.49% ±1.53 2.46 pts Reported ±1 SE ranges overlap
Code Migration v1.1 86.88% ±1.52 77.32% ±2.98 9.55 pts Reported ±1 SE ranges do not overlap
IOI 83.06% ±3.74 96.89% ±3.11 13.83 pts Reported ±1 SE ranges do not overlap
ProgramBench 6.50% ±1.75 3.00% ±1.21 3.50 pts Reported ±1 SE ranges do not overlap
SWE-bench Multimodal 58.33% ±2.25 43.54% ±2.27 14.79 pts Reported ±1 SE ranges do not overlap
Terminal-Bench 4.0 64.14% ±1.01 55.05% ±1.82 9.09 pts Reported ±1 SE ranges do not overlap
Vibe Code Bench v1.1 92.39% ±1.26 88.93% ±1.89 3.46 pts Reported ±1 SE ranges do not overlap
CyberBench v1.1 59.58% ±5.21 39.29% ±2.74 20.30 pts Reported ±1 SE ranges do not overlap
SRE Bench 30.15% ±2.84 50.76% ±3.10 20.61 pts Reported ±1 SE ranges do not overlap
Access to Justice Benchmark 74.94% ±0.70 65.84% ±0.76 9.11 pts Reported ±1 SE ranges do not overlap
Public Benefits Bench v1.1 67.19% ±1.22 59.34% ±1.28 7.85 pts Reported ±1 SE ranges do not overlap

Performance by category

Category Claude Sonnet 5.5 average GPT-6.1 Sol average
Legal 42.49% 37.51%
Finance 71.28% 63.66%
Healthcare 72.01% 67.65%
Math 100.00% 99.00%
Science 59.12% 59.63%
Education 88.03% 90.49%
Coding 65.22% 60.79%
Cyber 44.87% 45.02%
Social Mobility 71.06% 62.59%

Cost and latency

Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.

Benchmark Claude Sonnet 5.5 cost GPT-6.1 Sol cost Claude Sonnet 5.5 latency GPT-6.1 Sol latency
Vals Index $16.18 $3.61 1h26m 44m24s
Vals Finance Index $7.40 $1.36 43m31s 19m16s
Vals Legal Index $13.37 $2.75 1h15m 31m40s
Vals Science Index $8.50 $1.36 1h09m 33m11s
HLAB-Vals-1.1 $17.71 $3.54 1h13m 40m26s
Immigration $1.16 $0.23 11m03s 7m50s
Legal Research Bench $13.66 $3.38 1h33m 34m20s
ClaimBench $0.045922 $0.029239 4.05s 11.26s
EMB v1.1 $6.53 $1.14 36m45s 18m28s
Finance Agent v2.1 $6.95 $1.63 34m26s 19m05s
Tax Agent Bench $8.73 $1.32 59m21s 20m14s
MedCode N/A N/A 3m59s 115.34s
MedScribe N/A N/A 5m07s 2m50s
ProofBench v1.1 $0.52 $0.23 5m48s 4m45s
BioMysteryBench $2.95 $0.37 18m06s 8m51s
MysteryMechanism $2.98 $0.28 23m30s 7m17s
Terminal-Bench-Science $19.57 $3.44 2h46m 1h23m
SAGE $0.16 $0.08 81.12s 50.43s
Code Migration v1.1 $43.88 $9.51 4h07m 1h39m
IOI $5.81 $1.26 45m51s 31m31s
ProgramBench $22.62 $2.87 2h07m 1h10m
SWE-bench Multimodal $3.26 $0.35 24m13s 6m56s
Terminal-Bench 4.0 $10.75 $1.72 1h22m 45m12s
Vibe Code Bench v1.1 $19.73 $6.23 1h01m 1h15m
CyberBench v1.1 $2.91 $0.29 33m53s 3m42s
SRE Bench $17.27 $2.69 1h55m 32m32s
Access to Justice Benchmark $2.37 $0.69 17m34s 12m27s
Public Benefits Bench v1.1 $3.76 $0.69 1h15m 50m24s

Results available only for Claude Sonnet 5.5

None.

Results available only for GPT-6.1 Sol

  • Vals RSI Index
  • Robo-bench
Model details Claude Sonnet 5.5 Model details GPT-6.1 Sol