Public
18jeffreyma / Valkyrie
Benchmark updated: 8/18/202630 Tasks
Scalable, cloud-native infrastructure for evaluating AI agents across any benchmark.
Languages
Python99.5%Makefile0.4%Dockerfile<0.1%Mako<0.1%Shell<0.1%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 13 / 30 | $4.23 | $5/$25 | 13m23s |
Key Takeaways
- This single 30-task run provides no comparison with another model or harness.
- The run completes in 803.29 seconds on average; the result is directional rather than a statistically significant benchmark conclusion.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
No token usage data available.
Cost is the clearest tradeoff in this comparison. Claude Opus 4.8 leads at 43.33% for $4.23 per test. No other model in this comparison is cheaper.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
Claude Opus 4.8
Claude Opus 4.8 is both the most accurate and fastest model in this comparison at 43.33% and 13m 23s.