Public
pallets / click
Benchmark updated: 8/17/202630 Tasks
Python composable command line interface toolkit
Languages
Python100%Shell<0.1%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 27 / 30 | $1.65 | $5/$25 | 6m34s |
Key Takeaways
- This single 30-task run provides no comparison with other models or harnesses.
- Average latency was 393.84 seconds per test.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
No token usage data available.
Cost is the clearest tradeoff in this comparison. Claude Opus 5 leads at 90.00% for $1.65 per test. No other model in this comparison is cheaper.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Opus 5
Claude Opus 5 is both the most accurate and fastest model in this comparison at 90.00% and 6m 34s.