context rot
In plain wordsThe way an AI model becomes less reliable as you give it more and more text to read, even when the answer is in there somewhere.
The gradual loss of accuracy and instruction-following that language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. show as more material is packed into their context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out., even when the extra text leaves the task itself no harder. Unlike a hard context limit, it sets in unevenly: Chroma's 2025 measurements found it worse when the relevant passage was only loosely related in meaning to the question or when similar-looking distractors were present, and practitioners report it in long agent sessions full of tool output and dead ends. Proposed causes include attention being spread across ever more tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. (Anthropic speaks of a finite 'attention budget'), training data dominated by shorter sequences, and positional effects such as lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through.. Practical responses are context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else. techniques: retrieving less but better, compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there., clearing old tool results, and handing subtasks to subagentssubagent, n.: A helper AI agent that a main agent starts to handle one piece of a job; it works in its own separate workspace and reports back a short summary when it is done. with fresh contexts.
Past about 100k tokens the session showed classic context rot, so we compacted and restarted from a summary.
Origin
Earliest known use: a Hacker News comment of June 2025 by the user Workaccount2, describing output quality decaying as a conversation fills with distractions and dead ends, noticeable around 100,000 tokens with Gemini 2.5; Simon Willison quoted it on 18 June 2025. Chroma's technical report 'Context Rot' (Kelly Hong, Anton Troynikov and Jeff Huber, 14 July 2025), which measured the effect in 18 models, made it a standard term.
Usage note
Different from lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through. (sensitivity to where information sits) and from truncation once the window is full. Tests such as needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it. can understate context rot because they check literal lookup rather than reasoning over long inputs.
Related
- context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out.
- long contextlong context, n.: The ability of an AI model to work with very long inputs — whole books, codebases or hours of transcripts — in one go, and the techniques that make this possible.
- lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through.
- context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.
- compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there.
- needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it.
- context poisoningcontext poisoning, n.: When a mistake or a planted falsehood gets into an AI's working notes or memory and the AI keeps relying on it, so the error grows over later steps.
Sources
- Hong, K., Troynikov, A. & Huber, J. (2025). Context Rot: How Increasing Input Tokens Impacts LLM Performance. Chroma Technical Report, 14 July 2025. trychroma.com/research/context-rot
- Willison, S. (2025). Context rot (quoting Hacker News user Workaccount2). Simon Willison's Weblog, 18 June 2025. simonwillison.net/2025/Jun/18/context-rot
Mentioned in
context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out., compactioncompaction, n.: Like replacing a stack of rough notes with a one-page digest: when an AI's conversation grows too long for its memory, the history is condensed into a summary and the work continues from there., context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else., context poisoningcontext poisoning, n.: When a mistake or a planted falsehood gets into an AI's working notes or memory and the AI keeps relying on it, so the error grows over later steps., long contextlong context, n.: The ability of an AI model to work with very long inputs — whole books, codebases or hours of transcripts — in one go, and the techniques that make this possible., lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through., needle in a haystackneedle in a haystack, n.: A test for AI models that read very long documents: one unrelated sentence is hidden somewhere in a mass of text, and the model is asked about it., cache-augmented generationcache-augmented generation, n.: An alternative to search-based AI answering: load the whole knowledge base, if it is small enough, into the model's context once, save the model's processed state, and reuse it for every question., recursive language modelrecursive language model, n.: A way of running an AI language model so it can handle enormous documents by writing small programs to search and split them, and calling copies of itself on the pieces.