Muse Spark Open vs Mistral Large 4: Benchmark Comparison
Muse Spark Open has the higher score on 3 of 9 shared benchmarks; Mistral Large 4 leads on 6.
The largest observed score gap is 21.21 pts on Terminal-Bench 4.0 , where Mistral Large 4 leads.
Reported ±1 standard-error ranges overlap on 4 of 9 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Muse Spark Open | Mistral Large 4 | Gap | Reported uncertainty |
|---|---|---|---|---|
| Vals Finance Index | 52.99% ±1.44 | 56.90% ±1.42 | 3.91 pts | Reported ±1 SE ranges do not overlap |
| Vals Legal Index | 68.51% ±1.76 | 65.61% ±1.63 | 2.90 pts | Reported ±1 SE ranges overlap |
| HLAB-Vals-1.1 | 25.00% ±3.60 | 17.50% ±3.22 | 7.50 pts | Reported ±1 SE ranges do not overlap |
| Legal Research Bench | 38.46% ±3.38 | 31.73% ±3.23 | 6.73 pts | Reported ±1 SE ranges do not overlap |
| EMB v1.1 | 61.98% ±2.70 | 68.97% ±2.34 | 6.99 pts | Reported ±1 SE ranges do not overlap |
| Finance Agent v2.1 | 60.66% ±2.06 | 61.38% ±2.02 | 0.72 pts | Reported ±1 SE ranges overlap |
| Tax Agent Bench | 26.32% ±2.65 | 30.39% ±3.31 | 4.07 pts | Reported ±1 SE ranges overlap |
| Terminal-Bench 4.0 | 6.57% ±0.51 | 27.78% ±1.82 | 21.21 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench v1.1 | 76.19% ±3.32 | 78.40% ±3.55 | 2.22 pts | Reported ±1 SE ranges overlap |
Performance by category
| Category | Muse Spark Open average | Mistral Large 4 average |
|---|---|---|
| Legal | 31.73% | 24.62% |
| Finance | 49.66% | 53.58% |
| Coding | 41.38% | 53.09% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Muse Spark Open cost | Mistral Large 4 cost | Muse Spark Open latency | Mistral Large 4 latency |
|---|---|---|---|---|
| Vals Finance Index | $3.00 | $2.86 | 41m04s | 51m50s |
| Vals Legal Index | $5.36 | $4.08 | 1h33m | 1h09m |
| HLAB-Vals-1.1 | $13.63 | $4.18 | 3h59m | 58m04s |
| Legal Research Bench | $1.56 | $5.33 | 29m23s | 1h33m |
| EMB v1.1 | $5.35 | $4.84 | 1h25m | 1h22m |
| Finance Agent v2.1 | $2.76 | $1.02 | 26m52s | 16m26s |
| Tax Agent Bench | $0.88 | $2.72 | 11m42s | 57m18s |
| Terminal-Bench 4.0 | $7.26 | $10.25 | 2h04m | 1h40m |
| Vibe Code Bench v1.1 | $1.66 | $39.46 | 31m09s | 2h53m |
Results available only for Muse Spark Open
- Vals Index
- Code Migration v1.1
Results available only for Mistral Large 4
- MedCode
- MedScribe
- ProofBench v1.1
- BioMysteryBench
- MysteryMechanism
- IOI
- SWE-bench Multimodal
- Vibe Code Bench 1-100