Kimi K3 vs Step 5 Preview: Benchmark Comparison
Kimi K3 has the higher score on 11 of 14 shared benchmarks; Step 5 Preview leads on 3.
The largest observed score gap is 45.00 pts on ProofBench v1.1 , where Kimi K3 leads.
Reported ±1 standard-error ranges overlap on 8 of 14 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Kimi K3 | Step 5 Preview | Gap | Reported uncertainty |
|---|---|---|---|---|
| Vals Finance Index | 57.95% ±1.43 | 55.43% ±1.44 | 2.53 pts | Reported ±1 SE ranges overlap |
| Legal Research Bench | 46.15% ±3.46 | 40.38% ±3.41 | 5.77 pts | Reported ±1 SE ranges overlap |
| EMB v1.1 | 71.26% ±2.28 | 69.55% ±2.49 | 1.71 pts | Reported ±1 SE ranges overlap |
| Finance Agent v2.1 | 60.01% ±2.09 | 58.90% ±2.05 | 1.11 pts | Reported ±1 SE ranges overlap |
| Tax Agent Bench | 33.37% ±3.39 | 27.27% ±3.11 | 6.10 pts | Reported ±1 SE ranges overlap |
| ProofBench v1.1 | 87.00% ±3.38 | 42.00% ±4.96 | 45.00 pts | Reported ±1 SE ranges do not overlap |
| BioMysteryBench | 72.59% ±1.61 | 67.41% ±0.37 | 5.19 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench-Science | 1.43% ±1.43 | 2.86% ±2.01 | 1.43 pts | Reported ±1 SE ranges overlap |
| Code Migration v1.1 | 28.59% ±3.78 | 47.00% ±3.43 | 18.42 pts | Reported ±1 SE ranges do not overlap |
| IOI | 48.94% ±9.82 | 38.06% ±6.69 | 10.89 pts | Reported ±1 SE ranges overlap |
| SWE-bench Multimodal | 44.17% ±2.27 | 40.83% ±2.25 | 3.33 pts | Reported ±1 SE ranges overlap |
| Terminal-Bench 4.0 | 17.17% ±0.51 | 31.82% ±2.62 | 14.65 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench v1.1 | 84.97% ±2.75 | 70.40% ±4.27 | 14.57 pts | Reported ±1 SE ranges do not overlap |
| Public Benefits Bench v1.1 | 68.20% ±1.21 | 58.12% ±1.28 | 10.08 pts | Reported ±1 SE ranges do not overlap |
Performance by category
| Category | Kimi K3 average | Step 5 Preview average |
|---|---|---|
| Legal | 46.15% | 40.38% |
| Finance | 54.88% | 51.91% |
| Math | 87.00% | 42.00% |
| Science | 37.01% | 35.13% |
| Coding | 44.77% | 45.62% |
| Social Mobility | 68.20% | 58.12% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Kimi K3 cost | Step 5 Preview cost | Kimi K3 latency | Step 5 Preview latency |
|---|---|---|---|---|
| Vals Finance Index | $2.58 | $0.82 | 22m24s | 24m42s |
| Legal Research Bench | $3.47 | $0.84 | 12m40s | 36m08s |
| EMB v1.1 | $3.07 | $1.33 | 12m21s | 43m35s |
| Finance Agent v2.1 | $1.88 | $0.58 | 14m35s | 9m03s |
| Tax Agent Bench | $2.79 | $0.55 | 40m15s | 21m29s |
| ProofBench v1.1 | $1.65 | $0.71 | 34m31s | 38m34s |
| BioMysteryBench | $1.06 | $0.49 | 4m55s | 21m35s |
| Terminal-Bench-Science | $21.30 | $4.25 | 4h49m | 2h30m |
| Code Migration v1.1 | $35.75 | $8.61 | 4h31m | 3h17m |
| IOI | $14.17 | $2.03 | 4h18m | 1h20m |
| SWE-bench Multimodal | $1.65 | $0.61 | 22m45s | 26m06s |
| Terminal-Bench 4.0 | $12.48 | $3.11 | 3h09m | 1h58m |
| Vibe Code Bench v1.1 | $10.01 | $5.86 | 16m39s | 1h31m |
| Public Benefits Bench v1.1 | $0.91 | $0.50 | 7m19s | 27m19s |
Results available only for Kimi K3
- Vals Index
- Vals Inference Provider Index
- Vals Legal Index
- Vals RSI Index
- HLAB-Vals-1.1
- LegalBench
- MortgageTax
- TaxEval v2
- MedCode
- MedScribe
- GPQA Diamond
- MMLU Pro
- MMMU Pro
- SAGE
- LiveCodeBench
- ProgramBench
- SWE-bench
- Vibe Code Bench 1-100
- CyberBench v1.1
- Time Horizon Index: KSP
Results available only for Step 5 Preview
- Vals Science Index
- MysteryMechanism
- SRE Bench