Claude Opus 4.8 vs Gemini 3.8 Flash: Benchmark Comparison

Claude Opus 4.8 has the higher score on 11 of 20 shared benchmarks; Gemini 3.8 Flash leads on 8.

The largest observed score gap is 8.60 pts on SWE-bench , where Claude Opus 4.8 leads.

Reported ±1 standard-error ranges overlap on 11 of 19 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.

Shared benchmark results

Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.

Benchmark Claude Opus 4.8 Gemini 3.8 Flash Gap Reported uncertainty
Vals RSI Index 23.98% 19.93% 4.05 pts Uncertainty comparison unavailable
Legal Research Bench 43.75% ±3.45 38.94% ±3.39 4.81 pts Reported ±1 SE ranges overlap
LegalBench 83.57% ±0.47 86.99% ±0.43 3.42 pts Reported ±1 SE ranges do not overlap
MortgageTax 69.91% ±0.89 65.34% ±0.94 4.57 pts Reported ±1 SE ranges do not overlap
Tax Agent Bench 30.00% ±3.30 32.12% ±3.37 2.13 pts Reported ±1 SE ranges overlap
TaxEval v2 75.63% ±0.84 74.45% ±0.85 1.19 pts Reported ±1 SE ranges overlap
MedCode 53.22% ±2.17 48.13% ±2.18 5.08 pts Reported ±1 SE ranges do not overlap
MedScribe 85.75% ±1.93 84.50% ±1.94 1.26 pts Reported ±1 SE ranges overlap
Terminal-Bench-Science 4.29% ±2.44 8.57% ±3.37 4.29 pts Reported ±1 SE ranges overlap
GPQA Diamond 92.42% ±1.98 94.44% ±1.48 2.02 pts Reported ±1 SE ranges overlap
MMLU Pro 89.58% ±0.30 90.22% ±0.29 0.64 pts Reported ±1 SE ranges do not overlap
MMMU Pro 86.59% ±0.82 89.08% ±0.75 2.48 pts Reported ±1 SE ranges do not overlap
LiveCodeBench 87.82% ±0.95 89.48% ±0.90 1.66 pts Reported ±1 SE ranges overlap
ProgramBench 1.00% ±0.70 1.00% ±0.70 0.00 pts Reported ±1 SE ranges overlap
SkillsBench 59.23% ±4.50 57.98% ±4.42 1.24 pts Reported ±1 SE ranges overlap
SWE-bench 88.60% ±1.42 80.00% ±1.79 8.60 pts Reported ±1 SE ranges do not overlap
Terminal-Bench 4.0 23.23% ±1.34 19.70% ±2.31 3.54 pts Reported ±1 SE ranges overlap
Vibe Code Bench v1.1 82.72% ±3.08 78.65% ±3.88 4.07 pts Reported ±1 SE ranges overlap
Public Benefits Bench v1.1 68.13% ±1.21 65.29% ±1.24 2.84 pts Reported ±1 SE ranges do not overlap
Time Horizon Index: KSP 11.83% ±0.00 18.83% ±0.00 7.00 pts Reported ±1 SE ranges do not overlap

Performance by category

Category Claude Opus 4.8 average Gemini 3.8 Flash average
Legal 63.66% 62.97%
Finance 58.52% 57.30%
Healthcare 69.49% 66.32%
Science 4.29% 8.57%
Academic 89.53% 91.25%
Coding 57.10% 54.47%
Social Mobility 68.13% 65.29%

Cost and latency

Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.

Benchmark Claude Opus 4.8 cost Gemini 3.8 Flash cost Claude Opus 4.8 latency Gemini 3.8 Flash latency
Vals RSI Index $465.25 $332.04 90h00m 90h00m
Legal Research Bench $2.82 $1.63 11m54s 5m24s
LegalBench N/A N/A 3.11s 3.32s
MortgageTax N/A N/A 20.81s 14.79s
Tax Agent Bench $1.69 $0.86 7m59s 2m58s
TaxEval v2 N/A N/A 70.33s 8.76s
MedCode N/A N/A 105.92s 42.89s
MedScribe N/A N/A 82.36s 21.14s
Terminal-Bench-Science $23.14 $5.64 1h55m 55m11s
GPQA Diamond N/A N/A 2m14s 20.18s
MMLU Pro N/A N/A 23.44s 7.82s
MMMU Pro N/A N/A 49.04s 12.03s
LiveCodeBench N/A N/A 2m40s 24.07s
ProgramBench $31.27 $10.93 1h32m 46m39s
SkillsBench $4.77 $2.34 15m30s 3m55s
SWE-bench $1.92 $2.19 9m27s 11m23s
Terminal-Bench 4.0 $17.14 $8.65 1h11m 1h47m
Vibe Code Bench v1.1 $26.88 $6.87 1h16m 8m39s
Public Benefits Bench v1.1 $1.89 $1.01 12m48s 3m21s
Time Horizon Index: KSP $1499.32 $543.14 N/A N/A

Results available only for Claude Opus 4.8

None.

Results available only for Gemini 3.8 Flash

  • Vals Index
  • Vals Finance Index
  • Vals Legal Index
  • Vals Science Index
  • HLAB-Vals-1.1
  • Immigration
  • EMB v1.1
  • Finance Agent v2.1
  • ProofBench v1.1
  • BioMysteryBench
  • MysteryMechanism
  • SAGE
  • Code Migration v1.1
  • IOI
  • SWE-bench Multimodal
  • CyberBench v1.1
  • SRE Bench
  • Access to Justice Benchmark
  • Financial Coaching Bench
  • CUA-bench
Model details Claude Opus 4.8 Model details Gemini 3.8 Flash