Muse Spark 1.2 vs Step 5 Preview: Benchmark Comparison
Muse Spark 1.2 has the higher score on 5 of 9 shared benchmarks; Step 5 Preview leads on 4.
The largest observed score gap is 25.76 pts on Terminal-Bench 4.0 , where Step 5 Preview leads.
Reported ±1 standard-error ranges overlap on 4 of 9 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Muse Spark 1.2 | Step 5 Preview | Gap | Reported uncertainty |
|---|---|---|---|---|
| Legal Research Bench | 43.75% ±3.45 | 40.38% ±3.41 | 3.37 pts | Reported ±1 SE ranges overlap |
| Tax Agent Bench | 32.80% ±2.75 | 27.27% ±3.11 | 5.53 pts | Reported ±1 SE ranges overlap |
| ProofBench v1.1 | 43.00% ±4.98 | 42.00% ±4.96 | 1.00 pts | Reported ±1 SE ranges overlap |
| BioMysteryBench | 64.81% ±1.33 | 67.41% ±0.37 | 2.59 pts | Reported ±1 SE ranges do not overlap |
| IOI | 21.78% ±0.87 | 38.06% ±6.69 | 16.28 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench 4.0 | 6.06% ±1.51 | 31.82% ±2.62 | 25.76 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench v1.1 | 79.10% ±3.31 | 70.40% ±4.27 | 8.70 pts | Reported ±1 SE ranges do not overlap |
| SRE Bench | 0.00% ±0.00 | 0.38% ±0.38 | 0.38 pts | Reported ±1 SE ranges overlap |
| Public Benefits Bench v1.1 | 68.47% ±1.21 | 58.12% ±1.28 | 10.35 pts | Reported ±1 SE ranges do not overlap |
Performance by category
| Category | Muse Spark 1.2 average | Step 5 Preview average |
|---|---|---|
| Legal | 43.75% | 40.38% |
| Finance | 32.80% | 27.27% |
| Math | 43.00% | 42.00% |
| Science | 64.81% | 67.41% |
| Coding | 35.65% | 46.76% |
| Cyber | 0.00% | 0.38% |
| Social Mobility | 68.47% | 58.12% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Muse Spark 1.2 cost | Step 5 Preview cost | Muse Spark 1.2 latency | Step 5 Preview latency |
|---|---|---|---|---|
| Legal Research Bench | $0.51 | $0.84 | 5m07s | 36m08s |
| Tax Agent Bench | $0.27 | $0.55 | 119.91s | 21m29s |
| ProofBench v1.1 | $0.44 | $0.71 | 7m49s | 38m34s |
| BioMysteryBench | $0.99 | $0.49 | 23m37s | 21m35s |
| IOI | $2.64 | $2.03 | 25m35s | 1h20m |
| Terminal-Bench 4.0 | $4.14 | $3.11 | 1h18m | 1h58m |
| Vibe Code Bench v1.1 | $1.53 | $5.86 | 19m55s | 1h31m |
| SRE Bench | $23.43 | $3.07 | 1h40m | 1h34m |
| Public Benefits Bench v1.1 | $0.59 | $0.50 | 8m55s | 27m19s |
Results available only for Muse Spark 1.2
- LegalBench
- MortgageTax
- TaxEval v2
- MedCode
- MedScribe
- MMLU Pro
- MMMU Pro
- SAGE
- ProgramBench
- SkillsBench
- SWE-bench
- CyberBench v1.1
Results available only for Step 5 Preview
- Vals Finance Index
- Vals Science Index
- EMB v1.1
- Finance Agent v2.1
- MysteryMechanism
- Terminal-Bench-Science
- Code Migration v1.1
- SWE-bench Multimodal