DeepSeek V4.1 Flash vs Kimi K3: Benchmark Comparison

DeepSeek V4.1 Flash has the higher score on 6 of 24 shared benchmarks; Kimi K3 leads on 18.

The largest observed score gap is 33.00 pts on ProofBench v1.1 , where Kimi K3 leads.

Reported ±1 standard-error ranges overlap on 13 of 24 shared benchmarks with comparable uncertainty data. This is not a pairwise statistical significance test.

Shared benchmark results

Scores and reported standard errors come from the latest visible Vals benchmark version available for both models.

Benchmark DeepSeek V4.1 Flash Kimi K3 Gap Reported uncertainty
Vals Finance Index 57.08% ±1.45 57.95% ±1.43 0.88 pts Reported ±1 SE ranges overlap
Vals Inference Provider Index 66.60% ±3.25 64.85% ±3.20 1.74 pts Reported ±1 SE ranges overlap
Immigration 65.64% ±1.64 68.26% ±1.51 2.62 pts Reported ±1 SE ranges overlap
Legal Research Bench 41.35% ±3.42 46.15% ±3.46 4.81 pts Reported ±1 SE ranges overlap
LegalBench 83.28% ±0.46 86.21% ±0.41 2.94 pts Reported ±1 SE ranges do not overlap
EMB v1.1 67.76% ±2.56 71.26% ±2.28 3.51 pts Reported ±1 SE ranges overlap
Finance Agent v2.1 65.43% ±2.02 60.01% ±2.09 5.41 pts Reported ±1 SE ranges do not overlap
Tax Agent Bench 26.63% ±3.14 33.37% ±3.39 6.74 pts Reported ±1 SE ranges do not overlap
MedCode 41.17% ±2.04 49.36% ±2.20 8.18 pts Reported ±1 SE ranges do not overlap
MedScribe 85.50% ±1.92 88.05% ±1.98 2.55 pts Reported ±1 SE ranges overlap
ProofBench v1.1 54.00% ±5.01 87.00% ±3.38 33.00 pts Reported ±1 SE ranges do not overlap
BioMysteryBench 67.78% ±1.11 72.59% ±1.61 4.81 pts Reported ±1 SE ranges do not overlap
Terminal-Bench-Science 0.00% ±0.00 1.43% ±1.43 1.43 pts Reported ±1 SE ranges overlap
SAGE 85.38% ±1.96 86.77% ±1.85 1.39 pts Reported ±1 SE ranges overlap
Code Migration v1.1 58.94% ±2.66 28.59% ±3.78 30.35 pts Reported ±1 SE ranges do not overlap
IOI 40.28% ±2.56 48.94% ±9.82 8.67 pts Reported ±1 SE ranges overlap
ProgramBench 0.50% ±0.50 2.00% ±0.99 1.50 pts Reported ±1 SE ranges do not overlap
SWE-bench Multimodal 46.46% ±2.28 44.17% ±2.27 2.29 pts Reported ±1 SE ranges overlap
Terminal-Bench 4.0 19.70% ±1.75 17.17% ±0.51 2.52 pts Reported ±1 SE ranges do not overlap
Vibe Code Bench v1.1 84.74% ±2.89 84.97% ±2.75 0.23 pts Reported ±1 SE ranges overlap
CyberBench v1.1 73.69% ±5.48 75.24% ±5.56 1.55 pts Reported ±1 SE ranges overlap
Access to Justice Benchmark 72.86% ±0.72 68.17% ±0.75 4.69 pts Reported ±1 SE ranges do not overlap
Financial Coaching Bench 72.83% ±1.57 74.53% ±1.49 1.69 pts Reported ±1 SE ranges overlap
Public Benefits Bench v1.1 64.28% ±1.25 68.20% ±1.21 3.92 pts Reported ±1 SE ranges do not overlap

Performance by category

Category DeepSeek V4.1 Flash average Kimi K3 average
Legal 63.42% 66.87%
Finance 53.27% 54.88%
Healthcare 63.34% 68.70%
Math 54.00% 87.00%
Science 33.89% 37.01%
Education 85.38% 86.77%
Coding 41.77% 37.64%
Cyber 73.69% 75.24%
Social Mobility 69.99% 70.30%

Cost and latency

Cost per test appears only where the benchmark reports it for both models. Latency is the measured completion time for that benchmark.

Benchmark DeepSeek V4.1 Flash cost Kimi K3 cost DeepSeek V4.1 Flash latency Kimi K3 latency
Vals Finance Index $0.25 $2.58 11m37s 22m24s
Vals Inference Provider Index $0.57 $7.24 25m19s 55m01s
Immigration $0.05 $0.18 3m45s 64.84s
Legal Research Bench $0.25 $3.47 11m42s 12m40s
LegalBench N/A N/A 6.53s 7.11s
EMB v1.1 $0.23 $3.07 13m00s 12m21s
Finance Agent v2.1 $0.37 $1.88 16m00s 14m35s
Tax Agent Bench $0.13 $2.79 5m49s 40m15s
MedCode N/A N/A 35.13s 36.99s
MedScribe N/A N/A 53.83s 38.24s
ProofBench v1.1 $0.13 $1.65 9m07s 34m31s
BioMysteryBench $0.23 $1.06 14m50s 4m55s
Terminal-Bench-Science $0.47 $21.30 2h57m 4h49m
SAGE $0.01 $0.23 33.11s 2m25s
Code Migration v1.1 $1.09 $35.75 1h54m 4h31m
IOI $0.24 $14.17 18m30s 4h18m
ProgramBench $0.90 $70.48 1h02m 5h42m
SWE-bench Multimodal $0.23 $1.65 19m24s 22m45s
Terminal-Bench 4.0 $0.50 $12.48 48m22s 3h09m
Vibe Code Bench v1.1 $0.41 $10.01 15m28s 16m39s
CyberBench v1.1 $0.11 $2.13 13m50s 30m44s
Access to Justice Benchmark $0.11 $0.45 9m20s 9m42s
Financial Coaching Bench $0.02 $0.19 6m08s 8m29s
Public Benefits Bench v1.1 $0.07 $0.91 9m59s 7m19s

Results available only for DeepSeek V4.1 Flash

  • Vals Science Index
  • MysteryMechanism
  • SkillsBench
  • SRE Bench

Results available only for Kimi K3

  • Vals Index
  • Vals RSI Index
  • Vals Legal Index
  • HLAB-Vals-1.1
  • MortgageTax
  • TaxEval v2
  • GPQA Diamond
  • MMLU Pro
  • MMMU Pro
  • LiveCodeBench
  • SWE-bench
  • CyberOffense
  • Time Horizon Index: KSP
Model details DeepSeek V4.1 Flash Model details Kimi K3