Public
pingdotgg / t3code
Benchmark updated: 11/6/202686 Tasks
Custom repository benchmark.
Languages
TypeScript96.8%MDX1.8%Astro0.3%CSS0.2%C0.2%JavaScript0.2%Other0.6%
Harness | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
1 | 88.41% | 32.03% | 85.26% | $6.70 | 33m26s | |||||
2 | 74.09% | 34.15% | 83.97% | $3.04 | 30m47s | |||||
3 | 56.13% | 35.50% | 84.62% | $0.52 | 14m31s | |||||
4 | 38.77% | 27.04% | 50.00% | $0.06 | 14m05s |
Cost Analysis
Cost / Test vs. Accuracy
ACCURACYCOST
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.
Average Token Use / Test
Token Usage
InputOutputReasoningCache readCache write
Claude Opus 5.5
Claude Sonnet 5.5
GPT-6 Luna (Extra High)
GPT-6.1 Sol (Extra High)
Cost is the clearest tradeoff in this comparison. Claude Opus 5.5 · Pi leads at 68.57% for $6.70 per test. Claude Sonnet 5.5 · Pi is the lower-cost option at 64.07% for $3.04 per test.
Latency Analysis
Latency vs. Accuracy
ACCURACYLATENCY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.
Average Response Time / Test
Response Time
Claude Opus 5.5
Claude Sonnet 5.5
GPT-6.1 Sol (Extra High)
GPT-6 Luna (Extra High)
Latency separates several models with similarly strong scores. Claude Opus 5.5 · Pi leads at 68.57%, while GPT-6 Luna (Extra High) · Pi is fastest at 14m 5s with 38.60% accuracy.