Public

pingdotgg / t3code

Benchmark updated: 11/6/202686 Tasks
Add models

Custom repository benchmark.

Languages

TypeScript96.8%MDX1.8%Astro0.3%CSS0.2%C0.2%JavaScript0.2%Other0.6%

Harness

1

Pi

88.41%

32.03%

85.26%

$6.70
33m26s

2

Pi

74.09%

34.15%

83.97%

$3.04
30m47s

3

Pi

56.13%

35.50%

84.62%

$0.52
14m31s

4

Pi

38.77%

27.04%

50.00%

$0.06
14m05s

Cost Analysis

Cost / Test vs. Accuracy
ACCURACYCOST
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

Average Token Use / Test

Token Usage
InputOutputReasoningCache readCache write
Claude Opus 5.5
14.1M
Claude Sonnet 5.5
12.2M
GPT-6 Luna (Extra High)
3.6M
GPT-6.1 Sol (Extra High)
1.5M

Cost is the clearest tradeoff in this comparison. Claude Opus 5.5 · Pi leads at 68.57% for $6.70 per test. Claude Sonnet 5.5 · Pi is the lower-cost option at 64.07% for $3.04 per test.

Latency Analysis

Latency vs. Accuracy
ACCURACYLATENCY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

Average Response Time / Test

Response Time
Claude Opus 5.5
33m 26s
Claude Sonnet 5.5
30m 47s
GPT-6.1 Sol (Extra High)
14m 31s
GPT-6 Luna (Extra High)
14m 5s

Latency separates several models with similarly strong scores. Claude Opus 5.5 · Pi leads at 68.57%, while GPT-6 Luna (Extra High) · Pi is fastest at 14m 5s with 38.60% accuracy.