Public
suitedaces / dorabot
Benchmark updated: 8/18/202630 Tasks
macOS app for 24/7 AI agents in an IDE with memory, scheduled tasks, browser use + access to Whatsapp, Telegram, Slack.
Languages
TypeScript97.2%CSS1.8%JavaScript0.8%Shell0.1%HTML<0.1%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 20 / 30 | $0.52 | $1.25/$4.25 | 3m43s | ||
2 | Mini-SWE-agent | 19 / 30 | $0.03 | $0.44/$1.32 | 3m34s | ||
3 | Mini-SWE-agent | 19 / 30 | $0.47 | $1.4/$4.4 | 7m50s | ||
4 | Mini-SWE-agent | 19 / 30 | $2.37 | $5/$25 | 8m22s |
Key Takeaways
- Deepseek V4 Flash 0731 with Mini-SWE-agent matches the 19-task results at the lowest cost per test, $0.03, and lowest latency, 214.01 seconds.
- Muse Spark 1.2 with Mini-SWE-agent scores 66.67%, compared with 63.33% for each of the three 19-task results.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Muse Spark 1.2
Claude Opus 4.8
GLM 5.2
DeepSeek V4 Flash 0731
Cost is the clearest tradeoff in this comparison. Muse Spark 1.2 leads at 66.67% for $0.52 per test. DeepSeek V4 Flash 0731 is the lower-cost option at 63.33% for $0.03 per test.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Opus 4.8
GLM 5.2
Muse Spark 1.2
DeepSeek V4 Flash 0731
Latency separates several models with similarly strong scores. Muse Spark 1.2 leads at 66.67%, while DeepSeek V4 Flash 0731 is fastest at 3m 34s with 63.33% accuracy.