just-in-time retrieval
In plain wordsLetting an AI agent fetch information only at the moment it needs it — opening a file, running a search — instead of stuffing everything into its prompt in advance.
A context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else. strategy, so named in Anthropic's 2025 guidance on agents, in which an AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. holds only pointers in its contextcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out. — file paths, URLs, saved queries, document titles — and uses toolstool, n.: Any outside capability an AI model is allowed to use — a web search, a calculator, a code runner, a database or an app — described to the model so it can ask for it by name. such as search, file reading, shell utilities (grep, head, tail) or API calls to pull in the content behind a pointer at the moment a step needs it. It contrasts with up-front retrieval, where a pipeline embeds the query and inserts top-ranked chunks before the model starts. Just-in-time retrieval keeps context small and current and lets the agent exploit structure such as folder layouts and file names, at the cost of more tool callstool call, n.: A single request from an AI model to use one of its tools — naming the tool and filling in the details — which the surrounding software then carries out and reports back on., more latency and dependence on the agent's search skill. Many harnesses mix the two, loading a few standing files up front and fetching the rest on demand.
Instead of indexing the repository, the coding agent used grep just in time and read only the three files it needed.
Origin
The name borrows from just-in-time manufacturing and just-in-time compilation. Research systems had already retrieved on demand during generation (for example FLARE and Self-RAG, 2023); 'just in time' as the name for agentic, tool-driven context loading was popularized by Anthropic's 'Effective context engineering for AI agents' (29 September 2025), which cited Claude Code's use of glob and grep.
Related
- context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else.
- progressive disclosureprogressive disclosure, n.: Showing only a short summary first and revealing details when they are actually needed. AI agents use it to avoid filling their limited memory with instructions they may never use.
- retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.
- agentic RAGagentic RAG, n.: Retrieval in which the AI runs its own research: it decides what to search for, looks at the results, searches again if needed, and only then answers.
- tool usetool use, n.: An AI model's ability to reach beyond its own knowledge by operating outside tools — searching the web, running code, using a calculator or an app — and folding the results into its work.
- coding agentcoding agent, n.: An AI agent that writes and fixes software by itself: it reads a codebase, edits files, runs commands and tests, and keeps working until a feature is built or a bug is fixed.
Sources
- Rajasekaran, P., Dixon, E., Ryan, C. & Hadfield, J. (2025). Effective context engineering for AI agents. Anthropic Engineering blog, 29 September 2025. anthropic.com/engineering/effective-context-engineering-for-ai-agents
Mentioned in
agentic RAGagentic RAG, n.: Retrieval in which the AI runs its own research: it decides what to search for, looks at the results, searches again if needed, and only then answers., context engineeringcontext engineering, n.: Managing an AI model's reading material: choosing, trimming and arranging the instructions, documents, tool results and memories it is given on every turn, so that it has what the task needs and little else., progressive disclosureprogressive disclosure, n.: Showing only a short summary first and revealing details when they are actually needed. AI agents use it to avoid filling their limited memory with instructions they may never use.