Mercury 2.5 vs MiMo V2.5 Pro: Benchmark Comparison
Mercury 2.5 has the higher score on 1 of 8 shared benchmarks; MiMo V2.5 Pro leads on 7.
The largest observed score gap is 30.43 pts on Vibe Code Bench v1.1 , where MiMo V2.5 Pro leads.
Reported ±1 standard-error ranges overlap on 2 of 8 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Mercury 2.5 | MiMo V2.5 Pro | Gap | Reported uncertainty |
|---|---|---|---|---|
| Legal Research Bench | 4.33% ±1.41 | 15.87% ±2.54 | 11.54 pts | Reported ±1 SE ranges do not overlap |
| LegalBench | 83.06% ±0.48 | 77.13% ±0.55 | 5.93 pts | Reported ±1 SE ranges do not overlap |
| Tax Agent Bench | 2.49% ±1.02 | 11.50% ±1.96 | 9.02 pts | Reported ±1 SE ranges do not overlap |
| MedCode | 31.33% ±1.95 | 32.48% ±1.91 | 1.16 pts | Reported ±1 SE ranges overlap |
| MedScribe | 55.09% ±2.10 | 83.73% ±2.06 | 28.64 pts | Reported ±1 SE ranges do not overlap |
| ProofBench v1.1 | 3.00% ±1.71 | 22.00% ±4.16 | 19.00 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench 4.0 | 0.00% ±0.00 | 0.51% ±0.51 | 0.51 pts | Reported ±1 SE ranges overlap |
| Vibe Code Bench v1.1 | 3.68% ±1.41 | 34.11% ±4.53 | 30.43 pts | Reported ±1 SE ranges do not overlap |
Performance by category
| Category | Mercury 2.5 average | MiMo V2.5 Pro average |
|---|---|---|
| Legal | 43.69% | 46.50% |
| Finance | 2.49% | 11.50% |
| Healthcare | 43.21% | 58.11% |
| Math | 3.00% | 22.00% |
| Coding | 1.84% | 17.31% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Mercury 2.5 cost | MiMo V2.5 Pro cost | Mercury 2.5 latency | MiMo V2.5 Pro latency |
|---|---|---|---|---|
| Legal Research Bench | $0.06 | $0.08 | 47.61s | 6m56s |
| LegalBench | N/A | N/A | 3.91s | 27.14s |
| Tax Agent Bench | $0.02 | $0.04 | 29.67s | 6m17s |
| MedCode | N/A | N/A | 7.93s | 38.45s |
| MedScribe | N/A | N/A | 9.99s | 90.71s |
| ProofBench v1.1 | $0.04 | $0.09 | 3m22s | 19m37s |
| Terminal-Bench 4.0 | $0.37 | $0.23 | 15m14s | 2h07m |
| Vibe Code Bench v1.1 | $0.38 | $0.17 | 3m13s | 40m04s |
Results available only for Mercury 2.5
- Terminal-Bench-Science
- IOI
- SkillsBench
- CyberBench v1.1
- Public Benefits Bench v1.1
Results available only for MiMo V2.5 Pro
- TaxEval v2
- GPQA Diamond
- MMLU Pro
- LiveCodeBench
- SWE-bench