Public
conjfrnk / blackbird
Benchmark updated: 8/13/202630 Tasks
A minimal, macOS-native terminal emulator.
Languages
Swift74.2%Rust20%Shell4.5%C0.7%Metal0.2%Python0.1%Other0.3%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 17 / 30 | $3.55 | $5/$25 | 11m29s | ||
2 | Mini-SWE-agent | 16 / 30 | $0.70 | $1.25/$4.25 | 5m52s | ||
3 | Mini-SWE-agent | 15 / 30 | $0.07 | $0.2/$1.2 | 4m03s |
Key Takeaways
- Muse Spark 1.2 with Mini-SWE-agent resolves 16 of 30 tasks, one behind Claude Opus 5 and one ahead of GPT-5.6 Luna.
- GPT-5.6 Luna with Mini-SWE-agent resolves 15 of 30 tasks at $0.07 per test, the lowest supplied cost.
- Claude Opus 5 with Mini-SWE-agent reaches 17 of 30 tasks at $3.55 per test, while Muse Spark 1.2 reaches 16 at $0.70.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Claude Opus 5
Muse Spark 1.2
Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 56.67% for $3.55 per test. Muse Spark 1.2 is the lower-cost option at 53.33% for $0.70 per test.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Opus 5
Muse Spark 1.2
GPT-5.6 Luna
Latency separates several models with similarly strong scores. Claude Opus 5 leads at 56.67%, while GPT-5.6 Luna is fastest at 4m 3s with 50.00% accuracy.