Muse Spark 1.2 vs Mistral Large 4: Benchmark Comparison
Muse Spark 1.2 has the higher score on 6 of 9 shared benchmarks; Mistral Large 4 leads on 3.
The largest observed score gap is 33.00 pts on ProofBench v1.1 , where Muse Spark 1.2 leads.
Reported ±1 standard-error ranges overlap on 3 of 9 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Muse Spark 1.2 | Mistral Large 4 | Gap | Reported uncertainty |
|---|---|---|---|---|
| Legal Research Bench | 43.75% ±3.45 | 31.73% ±3.23 | 12.02 pts | Reported ±1 SE ranges do not overlap |
| Tax Agent Bench | 32.80% ±2.75 | 30.39% ±3.31 | 2.41 pts | Reported ±1 SE ranges overlap |
| MedCode | 49.35% ±2.19 | 40.69% ±2.17 | 8.66 pts | Reported ±1 SE ranges do not overlap |
| MedScribe | 90.06% ±1.96 | 80.43% ±2.00 | 9.63 pts | Reported ±1 SE ranges do not overlap |
| ProofBench v1.1 | 43.00% ±4.98 | 10.00% ±3.02 | 33.00 pts | Reported ±1 SE ranges do not overlap |
| BioMysteryBench | 64.81% ±1.33 | 67.04% ±2.67 | 2.22 pts | Reported ±1 SE ranges overlap |
| IOI | 21.78% ±0.87 | 45.28% ±3.78 | 23.50 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench 4.0 | 6.06% ±1.51 | 27.78% ±1.82 | 21.72 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench v1.1 | 79.10% ±3.31 | 78.40% ±3.55 | 0.70 pts | Reported ±1 SE ranges overlap |
Performance by category
| Category | Muse Spark 1.2 average | Mistral Large 4 average |
|---|---|---|
| Legal | 43.75% | 31.73% |
| Finance | 32.80% | 30.39% |
| Healthcare | 69.70% | 60.56% |
| Math | 43.00% | 10.00% |
| Science | 64.81% | 67.04% |
| Coding | 35.65% | 50.49% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Muse Spark 1.2 cost | Mistral Large 4 cost | Muse Spark 1.2 latency | Mistral Large 4 latency |
|---|---|---|---|---|
| Legal Research Bench | $0.51 | $5.33 | 5m07s | 1h33m |
| Tax Agent Bench | $0.27 | $2.72 | 119.91s | 57m18s |
| MedCode | N/A | N/A | 60.17s | 68.68s |
| MedScribe | N/A | N/A | 61.46s | 2m05s |
| ProofBench v1.1 | $0.44 | $0.64 | 7m49s | 20m59s |
| BioMysteryBench | $0.99 | $2.07 | 23m37s | 25m32s |
| IOI | $2.64 | $5.39 | 25m35s | 1h24m |
| Terminal-Bench 4.0 | $4.14 | $10.25 | 1h18m | 1h40m |
| Vibe Code Bench v1.1 | $1.53 | $39.46 | 19m55s | 2h53m |
Results available only for Muse Spark 1.2
- LegalBench
- MortgageTax
- TaxEval v2
- MMLU Pro
- MMMU Pro
- SAGE
- ProgramBench
- SkillsBench
- SWE-bench
- CyberBench v1.1
- SRE Bench
- Public Benefits Bench v1.1
Results available only for Mistral Large 4
- Vals Finance Index
- Vals Legal Index
- HLAB-Vals-1.1
- EMB v1.1
- Finance Agent v2.1
- MysteryMechanism
- SWE-bench Multimodal
- Vibe Code Bench 1-100