Public
oven-sh / bun
Benchmark updated: 8/15/202648 Tasks
Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one.
Languages
Rust65.6%C++21.6%TypeScript8.5%C2.7%JavaScript1.3%Shell0.3%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 26 / 48 | $2.26 | $3/$15 | 16m45s | ||
2 | Mini-SWE-agent | 25 / 48 | $4.11 | $5/$25 | 15m57s | ||
3 | Mini-SWE-agent | 25 / 48 | $7.33 | $10/$50 | 17m16s | ||
4 | Mini-SWE-agent | 24 / 48 | $2.29 | $4/$20 | 8m34s | ||
5 | Mini-SWE-agent | 24 / 48 | $4.00 | $5/$25 | 13m00s | ||
6 | Mini-SWE-agent | 23 / 48 | $0.81 | $2/$12 | 4m12s | ||
7 | Mini-SWE-agent | 23 / 48 | $0.11 | $0.2/$1.2 | 4m59s | ||
8 | Mini-SWE-agent | 23 / 48 | $1.80 | $5/$30 | 6m01s | ||
9 | Mini-SWE-agent | 23 / 48 | $0.59 | $2/$6 | 7m42s | ||
10 | Mini-SWE-agent | 22 / 48 | $4.65 | $2/$10 | 21m07s | ||
11 | Mini-SWE-agent | 21 / 48 | $0.38 | $2/$12 | 2m37s | ||
12 | Mini-SWE-agent | 21 / 48 | $1.24 | $1.5/$9 | 5m27s | ||
13 | Mini-SWE-agent | 21 / 48 | $0.74 | $1.4/$4.4 | 15m37s | ||
14 | Mini-SWE-agent | 16 / 48 | $0.30 | $1/$5 | 4m13s |
Key Takeaways
- GPT-5.6 Luna with Mini-SWE-agent resolves 23 of 48 tasks for $0.11 per test, the lowest cost among the supplied runs.
- GPT-5.6 Sol with Mini-SWE-agent resolves 24 of 48 tasks in 514 seconds, matching Claude Opus 4.7 on tasks resolved but finishing faster.
- Claude Haiku 4.5 (Nonthinking) with Mini-SWE-agent resolves 16 of 48 tasks, the lowest score in this set.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Claude Sonnet 5
Claude Opus 4.7
Claude Opus 4.8
Claude Fable 5
GLM 5.2
Kimi K3
Gemini 3.5 Flash
GPT-5.6 Luna
GPT-5.6 Sol
Claude Haiku 4.5 (Nonthinking)
GPT 5.5
Gemini 3.1 Pro Preview (02/26)
Grok 4.5
GPT-5.6 Terra
Cost is the clearest tradeoff in this comparison. Kimi K3 leads at 54.17% for $2.26 per test. No other model in this comparison is cheaper.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Sonnet 5
Claude Fable 5
Kimi K3
Claude Opus 4.8
GLM 5.2
Claude Opus 4.7
GPT-5.6 Sol
Grok 4.5
GPT 5.5
Gemini 3.5 Flash
GPT-5.6 Luna
Claude Haiku 4.5 (Nonthinking)
Gemini 3.1 Pro Preview (02/26)
GPT-5.6 Terra
Latency separates several models with similarly strong scores. Kimi K3 leads at 54.17%, while GPT-5.6 Terra is fastest at 2m 37s with 43.75% accuracy.