Public
sashabaranov / go-openai
Benchmark updated: 8/24/202630 Tasks
OpenAI, GPT 5.6, GPT-Image-2, Whisper API clients for Go
Languages
Go100%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 29 / 30 | $1.21 | $5/$25 | 5m17s | ||
2 | Mini-SWE-agent | 25 / 30 | $0.67 | $4/$20 | 3m24s | ||
3 | Mini-SWE-agent | 25 / 30 | $0.15 | $1.4/$4.4 | 5m30s | ||
4 | Mini-SWE-agent | 24 / 30 | $0.01 | $0.44/$1.32 | 63.38s | ||
5 | Mini-SWE-agent | 23 / 30 | $0.04 | $0.2/$1.2 | 3m17s |
Key Takeaways
- Claude Opus 5 with Mini-SWE-agent scores 96.67% at $1.21 per test; this 30-task result is directional.
- GLM 5.2 (Fireworks) and GPT-5.6 Sol with Mini-SWE-agent tie at 25 of 30; GPT-5.6 Sol is faster, while GLM 5.2 (Fireworks) costs less.
- Deepseek V4 Flash 0731 with Mini-SWE-agent resolves 24 of 30 at $0.01 per test, ahead of GPT-5.6 Luna’s 23 at $0.04.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Claude Opus 5
GPT-5.6 Luna
GLM 5.2
GPT-5.6 Sol
DeepSeek V4 Flash 0731
Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 96.67% for $1.21 per test. DeepSeek V4 Flash 0731 is the lower-cost option at 80.00% for $0.01 per test.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
GLM 5.2
Claude Opus 5
GPT-5.6 Sol
GPT-5.6 Luna
DeepSeek V4 Flash 0731
Latency separates several models with similarly strong scores. Claude Opus 5 leads at 96.67%, while DeepSeek V4 Flash 0731 is fastest at 1m 3s with 80.00% accuracy.