What a run looks like
A few minutes from each of three scored trials, compressed into five seconds. Every frame is preserved footage, not a re-enactment.
Minecraft — Claude Opus 5, an hour in. It has already punched out logs and built a crafting table and a wooden pickaxe from recipes it knew, and it picked up cobblestone at minute 50; here it is back among the birches for more wood before the stone pickaxe it crafts at minute 97. It acted once every 36 seconds, so these five minutes hold about eight decisions. It went on to build a furnace and, like every model, never found iron.
SUPERHOT — GPT-6 Astra, thirty minutes in. Time only advances when the agent moves, so every pause is the model thinking and every burst is it committing: here it closes on a red figure, fires, and throws a weapon. Even with the clock stopped between actions, each one still has to land a shot on a target in a 3D scene. Astra cleared five of the 25 levels, more than any other model.
eFootball — GPT-6 Astra in one of the two Beginner matches it won, with the menus between them cut. The ball does not wait: Astra acted every 28 seconds on average and held a direction key 743 times over the trial to keep a player moving while it decided the next thing. It won twice at Beginner and never at Regular.
The held-out three are not shown, for the same reason they are not named.
Key Takeaways
- No model plays at human speed. Across every trial a model acted once every 15 to 59 seconds, 34 at the median, and sent 2 to 19 inputs a minute where a person sends several a second.
- Spatial reasoning is where the models separate. GPT-6 Astra is the only model to score in the hidden sandbox game and climbed furthest in SUPERHOT, and four of five models cleared the reflex game’s first world. The points come where the world holds still long enough to be read.
Background
CUA-bench asks whether an AI agent can play games people actually buy. Six commercial titles, three hours each, keyboard and mouse only. The agent sees the screen and nothing else: no game console, no save files, no memory inspection, no network calls to the game. Every verifier lives on the harness side, where the agent cannot reach it. Most agent evaluations hand the model a text interface, an API, a DOM or a game state; here it gets what a person gets, pixels in and keystrokes out, so the score measures computer use itself rather than the fit of a wrapper.
The suite is built in pairs. Three games are public: Minecraft, SUPERHOT and eFootball. Each is matched with a held-out game of the same genre — one sandbox, one first-person reflex game, one sports title — whose identity we do not publish. A lab can train on a named title. It cannot train on one it cannot name, and the paired design means the hidden half measures the same kind of play as the half everyone can see.
A common fear is that the moment a benchmark is created, the labs will train against it. A private set lets us detect if and when that happens without giving up the generalization of the eval. If a model’s public scores pull away from its held-out scores, the pairs will show it.
Each game has its own ladder of milestones worth 100 points in total, and a model’s overall score is the mean of its six games. Milestones are credited in order: a later rung never pays before the rungs under it.
| Public game | Ladder |
|---|---|
| Minecraft | 13 rungs: log, crafting table, wooden pickaxe, cobblestone, stone pickaxe, furnace, iron ore, iron ingot, kill a hostile mob, flint and steel, obsidian, portal, enter the Nether |
| SUPERHOT | 25 rungs, 4 points each: clear one story level |
| eFootball | 5 rungs: win matches at rising difficulty, Beginner through Professional |
The three held-out games are scored the same way, on ordered milestones that add up to 100 points, though their rung counts and point values differ. We do not publish those ladders for the same reason we do not name the games.
Latency is a deliberate part of the test, and eFootball shows it most. These games run in real time whether or not the agent is ready: the ball keeps moving while a model is still deciding what to do, so its response time counts against it as much as its judgment does. To score at all, a model has to act at something like the speed a person would.
Three hours is the whole budget for a game: reading the screen, learning the controls, failing, and retrying. It is not three hours of successful play.
Why games
Games are a good test of computer use for practical reasons. They are purely multimodal: pixels in, keyboard and mouse out, with no API to lean on. They are latency-constrained: the world keeps moving, so response time is part of the score. People are good at them: children learn to operate them quickly and without instruction. And we chose them because the skills they exercise, reading a scene, acting under time pressure, holding a plan across hours, are ones we expect other computer work to draw on: football demands real-time decisions and continuous control, Minecraft demands navigating a 3D world and remembering where things are.
We are also watching for continuous learning, which we already see signs of in our KSP benchmark. There, what a model learns compresses neatly into text, so it can write the lesson down and carry it into the next attempt. Video does not compress that way. How would a model get better at eFootball? It can win at Beginner — but can it turn what worked there into a win at Regular, and that into a win at Professional? Those wins are reflexes and timing, not an equation it can restate in a sentence. We expect this to be much harder for models, and to take much more work to improve.
Results
Scores are the mean of the six games, each out of 100. The per-game columns show where the points came from. The time column is each model’s average number of seconds between one action and the next, over everything it played.
Every keystroke, click and mouse move an agent sends is logged, so the play itself can be counted and not just the outcome. Across the whole board the field sends between two and nineteen inputs a minute.
What the agent does
Each model plays through the command-line harness its own vendor ships — Codex for OpenAI models, Claude Code for Anthropic models, Google’s computer-use agent for Gemini — running on a separate CPU sandbox that drives one GPU game desktop over MCP. The agent may use its own shell and Python for scratch work on what it saw on screen. That work shares the same three-hour clock.
Methodology
Every trial is capped at three hours of gameplay. A trial that is interrupted by infrastructure resumes from its own saved game, its own model conversation, and only the unused remainder of its original three hours. Interruptions are never retried as fresh attempts, and a setup failure never becomes a model’s zero.
Scores are recomputed after the fact from preserved evidence under one frozen rubric, not from anything a live run reported about itself:
- Native evidence for the four offline games, read from the game’s own state: the world and server state for one, the games’ own save files for others, a mission registry for another.
- Video evidence for the two online sports games, where no readable save exists. The full recording is scanned at 2 Hz for result screens; each candidate result is confirmed by three distinct source frames and, for a claimed win, an agreeing text reading of the on-screen header.
- Signed recordings. Every trial records continuously and the recorder signs the archive. Grading rebuilds the movie from durable chunks and verifies its hash before scanning a frame.
Missing evidence is a grading failure, never a zero. An unplayed or unfinished game is reported as unscored rather than converted into a numeric floor. The held-out games are graded by the same rubric and the same pipeline as the public ones; holding a title back changes what we publish, not how it is scored. Where the pipeline cannot decide a cell, a person reviews the preserved evidence and the review is disclosed here.
Costs are registry-priced estimates from each trial’s signed request ledger, summed over every attempt a model made, including failed and superseded ones. 75 request attempts never returned usage and are excluded, so each figure is a floor, not an invoice.
Each cell is one three-hour trial per model per game, so the table carries no error bars. One result is not from the automatic pipeline: GPT-6 Astra’s hidden sports trial was replayed after its original run’s model transport failed. The replay lost its game session to a disconnect part-way through and was resumed in the same conversation from the same save for only its unused time. Its recording shows twelve lost first-stage attempts; the final result screen, after another lost match, rendered both “Cleared!” and “Failed”, which the automatic reader refused to resolve. A review of the game’s own state found the account never advanced past the first stage, so the cell is 0, the rubric’s result for a clear that is not confirmed. That review, not the pipeline, made the call.