Academic

Terminal-Bench 2.1

Updated 8/13/2026

State-of-the-art set of difficult terminal-based tasks

Terminal-Bench 2.1Difficult terminal-based coding tasks
ACCURACY

Key Takeaways

  • GPT-5.6 Sol leads at 85.77%, just ahead of Claude Opus 5 (84.64%) — but Sol’s advantage is driven by the medium split (86.06% vs. 83.64%), not the hard split where they are tied at 84.44% with a sharp cliff to third place (74.44%). While eleven models reach 100% on easy, the median score on that split is 83.33%, so even the easy tasks still separate the field.
  • Note on the headline scores: both Fable 5’s and Opus 5’s runs used Claude Opus 4.8 as a refusal fallback; counting Opus 5’s nine affected passes as failures lowers it from 84.64% to 81.27% (toggle the split icon beside the score to compare).

Background

Terminal-Bench 2.1 is an open-source benchmark that is designed to test a model’s ability to navigate and complete tasks in a sandboxed terminal environment. This version of the benchmark uses 89 tasks with unique categories ranging from model training to system administration. These tasks scale in difficulty from easy to hard.

We chose to include Terminal-Bench because a) it is increasingly common for it to be reported by model providers, b) it reflects the real-world terminal tasks expected of software engineers, and c) it is quite challenging, with no model scoring above 50% on the hard tasks upon its initial release. Furthermore, agentic systems like Claude Code, Codex, and Cursor now rely heavily on executing terminal commands correctly.

This benchmark was developed by the Terminal-Bench community as an open-source effort; we’d like to thank the community for their efforts in building this benchmark and for helping us integrate it into our evaluation suite. If you’re interested in learning more about Terminal-Bench or want to contribute to the project, visit tbench.ai

Below is an example task (you can find the full details for this task in the open-source task registry).

Please train a fasttext model on the yelp data in the data/ folder.

The final model size needs to be less than 150MB but get at least 0.62 accuracy on a private test set that comes from the same yelp review distribution.

The model should be saved as /app/model.bin

Methodology

All models were benchmarked using the Terminus 2 harness. Unless otherwise specified, we use identical configuration and methodology to Terminus 2. All results reported are pass@1.

On submission, we run the model against the provided pytests—a model must pass all pytests to get any credit for a task.

Unlike Terminus 1, Terminus 2 does not use structured outputs to enforce a response schema. Model queries returning an invalid or missing JSON are retried with a warning.

Comparison to Original Terminal-Bench

This benchmark is similar in structure to the original Terminal-Bench - it features 80 terminal-based tasks, many of which (like our example!) also appear in Terminal-Bench 2.1. The most substantial differences are in evaluation methododology:

  • We ran the original Terminal-Bench on an EC2 instance, using docker containers. Following the Laude implementation, we run Terminal-Bench 2.1 remotely using daytona.
  • We ran the original Terminal-Bench using a turn limit. Again, following the Laude implementation, we run Terminal-Bench 2.1 using a time limit instead.