Public
nikil-ravi / ThunderKittens
Benchmark updated: 8/13/202630 Tasks
Tile primitives for speedy kernels
Languages
Cuda86.3%Python12.9%Makefile0.8%
Harness | Input / Output Cost | ||||||
|---|---|---|---|---|---|---|---|
1 |  Mini-SWE-agent | 16 / 30 | $0.07 | $0.2/$1.2 | 3m55s |
Key Takeaways
- This 30-task result is directional rather than a smoke test.
- GPT-5.6 Luna with Mini-SWE-agent records 235.10 seconds of latency per test.
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
GPT-5.6 Luna
Cost is the clearest tradeoff in this comparison. GPT-5.6 Luna leads at 53.33% for $0.07 per test. No other model in this comparison is cheaper.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Average Response Time / Test
Response Time
GPT-5.6 Luna
GPT-5.6 Luna is both the most accurate and fastest model in this comparison at 53.33% and 3m 55s.