Public

nationalsecurityagency / ghidra

Benchmark updated: 8/13/202630 Tasks
Add models

Ghidra is a software reverse engineering (SRE) framework

Languages

Java86%C++6%HTML4.1%C1.5%Python1.4%Shell0.3%Other0.6%

Harness

1

zAI
GLM 5.2
Mini-SWE-agent
22 / 30

$0.32

5m41s

2

SpaceXAI
Grok 4.5
Mini-SWE-agent
21 / 30

$0.20

2m12s

Key Takeaways

  • GLM 5.2 (Fireworks) with Mini-SWE-agent scores 73.33%, compared with 70% for Grok 4.5 with Mini-SWE-agent.
  • Grok 4.5 with Mini-SWE-agent costs $0.20 per test and takes 132.05 seconds, versus $0.32 and 341.16 seconds for GLM 5.2 (Fireworks) with Mini-SWE-agent.

Cost Analysis

Cost / Test vs. Accuracy
ACCURACYCOST

Average Token Use / Test

Token Usage
InputOutputReasoningCache readCache write
GLM 5.2
1.4M
Grok 4.5
324K

Cost is the clearest tradeoff in this comparison. GLM 5.2 leads at 73.33% for $0.32 per test. Grok 4.5 is the lower-cost option at 70.00% for $0.20 per test.

Latency Analysis

Latency vs. Accuracy
ACCURACYLATENCY

Average Response Time / Test

Response Time
GLM 5.2
5m 41s
Grok 4.5
2m 12s

Latency separates several models with similarly strong scores. GLM 5.2 leads at 73.33%, while Grok 4.5 is fastest at 2m 12s with 70.00% accuracy.