Public
arandomhooman / hoomans-morphe-patches
Benchmark updated: 8/18/202630 Tasks
Patches for morphe
Languages
Kotlin89.6%Java10.4%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 6 / 30 | $2.09 | $4/$20 | 11m38s | ||
2 | Mini-SWE-agent | 5 / 30 | $0.03 | $0.44/$1.32 | 4m55s | ||
3 | Mini-SWE-agent | 5 / 30 | $3.19 | $5/$25 | 12m32s | ||
4 | Mini-SWE-agent | 5 / 30 | $3.64 | $5/$25 | 15m15s |
Key Takeaways
- This is a directional result because the 30-task benchmark has multiple supplied runs but no repeated-run uncertainty metrics.
- Deepseek V4 Flash 0731 with Mini-SWE-agent matches Claude Opus 4.8 and Claude Opus 5 at 5 of 30 tasks while reporting the lowest cost per test, $0.03.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Claude Opus 4.8
Claude Opus 5
GPT-5.6 Sol
DeepSeek V4 Flash 0731
Cost is the clearest tradeoff in this comparison. GPT-5.6 Sol leads at 20.00% for $2.09 per test. DeepSeek V4 Flash 0731 is the lower-cost option at 16.67% for $0.03 per test.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Opus 4.8
Claude Opus 5
GPT-5.6 Sol
DeepSeek V4 Flash 0731
Latency separates several models with similarly strong scores. GPT-5.6 Sol leads at 20.00%, while DeepSeek V4 Flash 0731 is fastest at 4m 55s with 16.67% accuracy.