Claude Sonnet 5 vs Kimi K3: Benchmark Comparison

Claude Sonnet 5 has the higher score on 2 of 20 shared benchmarks; Kimi K3 leads on 18.

The largest observed score gap is 13.80 pts on SWE-bench , where Kimi K3 leads.

Reported ±1 standard-error ranges overlap on 8 of 20 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.

Shared benchmark results

Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.

Benchmark Claude Sonnet 5 Kimi K3 Gap Reported uncertainty
Legal Research Bench 41.83% ±3.43 46.15% ±3.46 4.33 pts Reported ±1 SE ranges overlap
LegalBench 83.92% ±0.46 86.21% ±0.41 2.29 pts Reported ±1 SE ranges do not overlap
MortgageTax 70.03% ±0.90 66.34% ±0.91 3.70 pts Reported ±1 SE ranges do not overlap
Tax Agent Bench 29.37% ±3.25 33.37% ±3.39 3.99 pts Reported ±1 SE ranges overlap
TaxEval v2 75.63% ±0.84 75.72% ±0.84 0.08 pts Reported ±1 SE ranges overlap
MedCode 47.54% ±2.27 49.36% ±2.20 1.82 pts Reported ±1 SE ranges overlap
MedScribe 76.05% ±3.05 88.05% ±1.98 11.99 pts Reported ±1 SE ranges do not overlap
ProofBench v1.1 77.00% ±4.23 87.00% ±3.38 10.00 pts Reported ±1 SE ranges do not overlap
Terminal-Bench-Science 5.71% ±2.79 1.43% ±1.43 4.29 pts Reported ±1 SE ranges do not overlap
GPQA Diamond 88.89% ±2.22 92.93% ±1.31 4.04 pts Reported ±1 SE ranges do not overlap
MMLU Pro 87.55% ±0.37 87.97% ±0.32 0.43 pts Reported ±1 SE ranges overlap
MMMU Pro 83.01% ±0.90 88.15% ±0.78 5.14 pts Reported ±1 SE ranges do not overlap
IOI 45.00% ±2.75 48.94% ±9.82 3.94 pts Reported ±1 SE ranges overlap
LiveCodeBench 82.43% ±1.09 87.19% ±0.97 4.76 pts Reported ±1 SE ranges do not overlap
ProgramBench 0.00% ±0.00 2.00% ±0.99 2.00 pts Reported ±1 SE ranges do not overlap
SWE-bench 79.60% ±1.80 93.40% ±1.11 13.80 pts Reported ±1 SE ranges do not overlap
Terminal-Bench 4.0 10.61% ±1.51 17.17% ±0.51 6.57 pts Reported ±1 SE ranges do not overlap
Vibe Code Bench v1.1 81.33% ±3.05 84.97% ±2.75 3.64 pts Reported ±1 SE ranges overlap
CyberBench v1.1 61.91% ±5.74 75.24% ±5.56 13.33 pts Reported ±1 SE ranges do not overlap
Public Benefits Bench v1.1 66.03% ±1.23 68.20% ±1.21 2.17 pts Reported ±1 SE ranges overlap

Performance by category

Category Claude Sonnet 5 average Kimi K3 average
Legal 62.87% 66.18%
Finance 58.35% 58.47%
Healthcare 61.80% 68.70%
Math 77.00% 87.00%
Science 5.71% 1.43%
Academic 86.48% 89.68%
Coding 49.83% 55.61%
Cyber 61.91% 75.24%
Social Mobility 66.03% 68.20%

Cost and latency

Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.

Benchmark Claude Sonnet 5 cost Kimi K3 cost Claude Sonnet 5 latency Kimi K3 latency
Legal Research Bench $2.72 $3.47 25m46s 12m40s
LegalBench N/A N/A 4.90s 7.11s
MortgageTax N/A N/A 28.67s 78.94s
Tax Agent Bench $1.79 $2.79 19m19s 40m15s
TaxEval v2 N/A N/A 3m22s 2m25s
MedCode N/A N/A 2m15s 36.99s
MedScribe N/A N/A 4m12s 38.24s
ProofBench v1.1 $1.37 $1.65 14m58s 34m31s
Terminal-Bench-Science $30.30 $21.30 3h06m 4h49m
GPQA Diamond N/A N/A 63.23s 2m02s
MMLU Pro N/A N/A 25.19s 38.72s
MMMU Pro N/A N/A 18.82s 106.52s
IOI $12.89 $14.17 59m44s 4h18m
LiveCodeBench N/A N/A 77.02s 3m20s
ProgramBench $24.36 $70.48 1h31m 5h42m
SWE-bench $1.49 $0.76 16m02s 10m19s
Terminal-Bench 4.0 $26.05 $12.48 1h45m 3h09m
Vibe Code Bench v1.1 $25.39 $10.01 1h07m 16m39s
CyberBench v1.1 $1.89 $2.13 16m57s 30m44s
Public Benefits Bench v1.1 $1.29 $0.91 21m09s 7m19s

Results available only for Claude Sonnet 5

  • SkillsBench
  • SRE Bench

Results available only for Kimi K3

  • Vals Index
  • Vals RSI Index
  • Vals Finance Index
  • Vals Legal Index
  • Vals Inference Provider Index
  • HLAB-Vals-1.1
  • Immigration
  • EMB v1.1
  • Finance Agent v2.1
  • BioMysteryBench
  • SAGE
  • Code Migration v1.1
  • SWE-bench Multimodal
  • CyberOffense
  • Access to Justice Benchmark
  • Financial Coaching Bench
  • Time Horizon Index: KSP
Model details Claude Sonnet 5 Model details Kimi K3