recursive language model

An inference strategy in which a language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. answers a query over very long input without reading it all into its context windowcontext window, n.: How much an AI model can read and keep in view at once: the instructions, the conversation so far and any attached documents. Text that does not fit has to be cut, summarized or left out.: the input is stored as a variable in a programming environment (a Python REPL), and a 'root' model writes code to inspect, search, partition and transform it, launching recursive calls to itself or to smaller sub-models on selected chunks before returning a final answer. To the user it behaves like an ordinary model call. In the introducing experiments, a recursive wrapper around a small model beat a much larger model on long-context reasoning benchmarks at similar or lower cost per query, and stayed accurate over document collections exceeding ten million tokenstoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens.. The approach treats context rotcontext rot, n.: The way an AI model becomes less reliable as you give it more and more text to read, even when the answer is in there somewhere. as an engineering problem to route around rather than a limit to be trained away.

We wrapped the small model as a recursive language model and it handled the whole 8,000-page discovery archive.