Step 5 Preview vs Hy4 Preview: Benchmark Comparison
Step 5 Preview has the higher score on 4 of 14 shared benchmarks; Hy4 Preview leads on 10.
The largest observed score gap is 33.00 pts on ProofBench v1.1 , where Hy4 Preview leads.
Reported ±1 standard-error ranges overlap on 9 of 14 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Step 5 Preview | Hy4 Preview | Gap | Reported uncertainty |
|---|---|---|---|---|
| Vals Finance Index | 55.43% ±1.44 | 57.51% ±1.45 | 2.08 pts | Reported ±1 SE ranges overlap |
| Legal Research Bench | 40.38% ±3.41 | 45.19% ±3.46 | 4.81 pts | Reported ±1 SE ranges overlap |
| EMB v1.1 | 69.55% ±2.49 | 68.95% ±2.40 | 0.61 pts | Reported ±1 SE ranges overlap |
| Finance Agent v2.1 | 58.90% ±2.05 | 61.62% ±2.06 | 2.72 pts | Reported ±1 SE ranges overlap |
| Tax Agent Bench | 27.27% ±3.11 | 32.62% ±3.38 | 5.34 pts | Reported ±1 SE ranges overlap |
| ProofBench v1.1 | 42.00% ±4.96 | 75.00% ±4.35 | 33.00 pts | Reported ±1 SE ranges do not overlap |
| BioMysteryBench | 67.41% ±0.37 | 69.26% ±2.59 | 1.85 pts | Reported ±1 SE ranges overlap |
| Terminal-Bench-Science | 2.86% ±2.01 | 1.43% ±1.43 | 1.43 pts | Reported ±1 SE ranges overlap |
| Code Migration v1.1 | 47.00% ±3.43 | 45.36% ±3.59 | 1.64 pts | Reported ±1 SE ranges overlap |
| IOI | 38.06% ±6.69 | 59.33% ±4.60 | 21.28 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench 4.0 | 31.82% ±2.62 | 8.08% ±1.34 | 23.74 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench v1.1 | 70.40% ±4.27 | 77.48% ±4.04 | 7.08 pts | Reported ±1 SE ranges overlap |
| SRE Bench | 0.38% ±0.38 | 2.29% ±0.93 | 1.91 pts | Reported ±1 SE ranges do not overlap |
| Public Benefits Bench v1.1 | 58.12% ±1.28 | 68.61% ±1.21 | 10.49 pts | Reported ±1 SE ranges do not overlap |
Performance by category
| Category | Step 5 Preview average | Hy4 Preview average |
|---|---|---|
| Legal | 40.38% | 45.19% |
| Finance | 51.91% | 54.40% |
| Math | 42.00% | 75.00% |
| Science | 35.13% | 35.34% |
| Coding | 46.82% | 47.56% |
| Cyber | 0.38% | 2.29% |
| Social Mobility | 58.12% | 68.61% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Step 5 Preview cost | Hy4 Preview cost | Step 5 Preview latency | Hy4 Preview latency |
|---|---|---|---|---|
| Vals Finance Index | $0.82 | $0.68 | 24m42s | 32m15s |
| Legal Research Bench | $0.84 | $0.85 | 36m08s | 1h04m |
| EMB v1.1 | $1.33 | $0.79 | 43m35s | 30m39s |
| Finance Agent v2.1 | $0.58 | $0.67 | 9m03s | 25m03s |
| Tax Agent Bench | $0.55 | $0.59 | 21m29s | 41m04s |
| ProofBench v1.1 | $0.71 | $0.34 | 38m34s | 22m54s |
| BioMysteryBench | $0.49 | $0.36 | 21m35s | 23m30s |
| Terminal-Bench-Science | $4.25 | $2.14 | 2h30m | 3h48m |
| Code Migration v1.1 | $8.61 | $3.58 | 3h17m | 1h54m |
| IOI | $2.03 | $1.35 | 1h20m | 1h01m |
| Terminal-Bench 4.0 | $3.11 | $2.10 | 1h58m | 2h10m |
| Vibe Code Bench v1.1 | $5.86 | $2.18 | 1h31m | 41m33s |
| SRE Bench | $3.07 | $4.90 | 1h34m | 1h40m |
| Public Benefits Bench v1.1 | $0.50 | $0.25 | 27m19s | 1h03m |
Results available only for Step 5 Preview
- Vals Science Index
- MysteryMechanism
- SWE-bench Multimodal
Results available only for Hy4 Preview
- Vals Index
- Vals Legal Index
- HLAB-Vals-1.1
- LegalBench
- MedCode
- MedScribe
- ProgramBench
- CyberBench v1.1