Public
arduino / arduino-ide
Benchmark updated: 8/14/202630 Tasks
Arduino IDE 2.x
Languages
TypeScript49.8%JavaScript48.4%CSS1.6%HTML0.2%Shell<0.1%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 13 / 30 | $0.03 | $0.2/$1.2 | 2m15s | ||
2 | Mini-SWE-agent | 12 / 30 | $1.91 | $1.5/$7.5 | 16m32s | ||
3 | Mini-SWE-agent | 10 / 30 | $0.64 | $1.4/$4.4 | 12m29s |
Key Takeaways
- GPT-5.6 Luna (High) with Mini-SWE-agent scores 43.33%, ahead of Gemini 3.6 Flash with Mini-SWE-agent at 40% and GLM 5.2 (Fireworks) with Mini-SWE-agent at 33.33%.
- GPT-5.6 Luna (High) with Mini-SWE-agent has the lowest cost per test at $0.03 and shortest latency at 135.11 seconds among the supplied runs.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Gemini 3.6 Flash
GLM 5.2
GPT-5.6 Luna
Cost is the clearest tradeoff in this comparison. GPT-5.6 Luna leads at 43.33% for $0.03 per test. No other model in this comparison is cheaper.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Gemini 3.6 Flash
GLM 5.2
GPT-5.6 Luna
GPT-5.6 Luna is both the most accurate and fastest model in this comparison at 43.33% and 2m 15s.