Qwen 3.8 Max vs Mistral Large 4: Benchmark Comparison
Qwen 3.8 Max has the higher score on 8 of 12 shared benchmarks; Mistral Large 4 leads on 4.
The largest observed score gap is 48.00 pts on ProofBench v1.1 , where Qwen 3.8 Max leads.
Reported ±1 standard-error ranges overlap on 4 of 12 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.
Shared benchmark results
Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.
| Benchmark | Qwen 3.8 Max | Mistral Large 4 | Gap | Reported uncertainty |
|---|---|---|---|---|
| Legal Research Bench | 47.60% ±3.47 | 31.73% ±3.23 | 15.86 pts | Reported ±1 SE ranges do not overlap |
| EMB v1.1 | 65.99% ±2.69 | 68.97% ±2.34 | 2.99 pts | Reported ±1 SE ranges overlap |
| Tax Agent Bench | 32.09% ±3.37 | 30.39% ±3.31 | 1.70 pts | Reported ±1 SE ranges overlap |
| MedCode | 40.67% ±2.03 | 40.69% ±2.17 | 0.02 pts | Reported ±1 SE ranges overlap |
| MedScribe | 84.95% ±2.00 | 80.43% ±2.00 | 4.51 pts | Reported ±1 SE ranges do not overlap |
| ProofBench v1.1 | 58.00% ±4.96 | 10.00% ±3.02 | 48.00 pts | Reported ±1 SE ranges do not overlap |
| MysteryMechanism | 23.87% ±2.87 | 12.16% ±2.20 | 11.71 pts | Reported ±1 SE ranges do not overlap |
| IOI | 68.89% ±5.20 | 45.28% ±3.78 | 23.61 pts | Reported ±1 SE ranges do not overlap |
| SWE-bench Multimodal | 42.50% ±2.26 | 33.13% ±2.15 | 9.38 pts | Reported ±1 SE ranges do not overlap |
| Terminal-Bench 4.0 | 34.34% ±3.94 | 27.78% ±1.82 | 6.57 pts | Reported ±1 SE ranges do not overlap |
| Vibe Code Bench 1-100 | 12.83% ±3.18 | 14.26% ±3.21 | 1.43 pts | Reported ±1 SE ranges overlap |
| Vibe Code Bench v1.1 | 64.70% ±5.36 | 78.40% ±3.55 | 13.71 pts | Reported ±1 SE ranges do not overlap |
Performance by category
| Category | Qwen 3.8 Max average | Mistral Large 4 average |
|---|---|---|
| Legal | 47.60% | 31.73% |
| Finance | 49.04% | 49.68% |
| Healthcare | 62.81% | 60.56% |
| Math | 58.00% | 10.00% |
| Science | 23.87% | 12.16% |
| Coding | 44.65% | 39.77% |
Cost and latency
Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.
| Benchmark | Qwen 3.8 Max cost | Mistral Large 4 cost | Qwen 3.8 Max latency | Mistral Large 4 latency |
|---|---|---|---|---|
| Legal Research Bench | $2.49 | $5.33 | 1h01m | 1h33m |
| EMB v1.1 | $2.66 | $4.84 | 57m47s | 1h22m |
| Tax Agent Bench | $1.55 | $2.72 | 40m36s | 57m18s |
| MedCode | N/A | N/A | 6m18s | 68.68s |
| MedScribe | N/A | N/A | 4m11s | 2m05s |
| ProofBench v1.1 | $1.50 | $0.64 | 31m08s | 20m59s |
| MysteryMechanism | $1.31 | $2.10 | 37m49s | 35m37s |
| IOI | $9.47 | $5.39 | 1h47m | 1h24m |
| SWE-bench Multimodal | $0.99 | $5.14 | 18m44s | 27m27s |
| Terminal-Bench 4.0 | $10.63 | $10.25 | 2h28m | 1h40m |
| Vibe Code Bench 1-100 | $9.68 | $6.75 | 1h36m | 59m53s |
| Vibe Code Bench v1.1 | $8.24 | $39.46 | 2h49m | 2h53m |
Results available only for Qwen 3.8 Max
- Vals RSI Index
- LegalBench
- MortgageTax
- TaxEval v2
- Terminal-Bench-Science
- GPQA Diamond
- MMLU Pro
- MMMU Pro
- SAGE
- LiveCodeBench
- ProgramBench
- SkillsBench
- SWE-bench
- CyberBench v1.1
- Public Benefits Bench v1.1
Results available only for Mistral Large 4
- Vals Finance Index
- Vals Legal Index
- HLAB-Vals-1.1
- Finance Agent v2.1
- BioMysteryBench