Claude Plays Pokémon
In plain wordsA 2025 livestream in which Anthropic's Claude AI played the classic Game Boy game Pokémon Red by itself, watched by thousands as it struggled and slowly progressed; it inspired similar runs by other models.
A livestreamed experiment in which a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. plays the 1996 Game Boy role-playing game Pokémon Red autonomously, receiving screenshots and game-state information and pressing buttons through tool callstool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on., with its reasoning shown to viewers; by extension, the informal genre of AI-plays-Pokémon runs used to gauge long-horizon agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. abilities. The runs exposed weaknesses that short benchmarks miss — poor spatial navigation from screenshots, loops lasting days, unreliable memory across a context that must repeatedly be summarized — and showed how much performance depends on the harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car.: navigation aids, memory tools and map information supplied to the model. Cross-model comparisons were therefore hard to interpret, a point raised when a Gemini model finished Pokémon Blue using a different, more supportive harness.
The model aced the coding benchmark, then spent two days on the Pokémon stream walking into the same wall.
Origin
Launched on Twitch by Anthropic in February 2025, alongside the release of Claude 3.7 Sonnet, whose announcement charted progress through the game as a measure of agentic ability; it echoed the crowd-controlled 'Twitch Plays Pokémon' of 2014. An independent developer's 'Gemini Plays Pokémon' stream completed Pokémon Blue with Gemini 2.5 Pro in May 2025.
Usage note
The genre is entertainment as much as evaluation: results depend heavily on harness design, which is rarely held constant across models.
Related
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- harnessharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car.
- agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.
- compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- jagged intelligencejagged intelligence, n.: The uneven pattern of AI abilities: today's systems can solve very hard problems yet fail at some a child could manage, so their skills form a spiky profile rather than a smooth one.