stop sequence
In plain wordsA piece of text that tells an AI model to stop writing as soon as it produces it — for example, stopping at a blank line or just before the next question.
A string that, when produced by a language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. during generation, ends the output immediately, usually with the string itself omitted from the returned text. Stop sequences complement the model's own end-of-sequence special tokenspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes. and the maximum-length limit, letting developers cut generation at a structural boundary — the end of a list item, a closing tag, a new 'User:' turn — so the model does not continue into content it should not write, such as the other side of a dialogue. APIs typically accept a handful of stop strings and report which stop condition ended generation. Because matching happens on decoded text while generation proceeds token by token, engines must detect stop strings that span tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. boundaries. Stop sequences were essential when using base modelsbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering. with few-shot promptsfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case. and remain useful in agent loopsagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished. and structured output.
Add '\nQ:' as a stop sequence so the model answers one question and halts.
Origin
A standard parameter of text-generation APIs since OpenAI's GPT-3 API (2020), whose stop parameter accepts up to four sequences; halting generation at a delimiter is an older practice in text generation.
Usage note
Not to be confused with stop words, the very common words that some search and NLP pipelines discard, or with an agent's stop conditionstop condition, n.: The rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong..
Related
- decodingdecoding, n.: The method a language model uses to turn its predictions into actual text — for example, always picking the most likely next word, or choosing somewhat at random among the likely ones.
- special tokenspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes.
- base modelbase model, n.: A language model fresh from its first broad training on text, before being taught to follow instructions or chat. It continues whatever text it is given instead of answering.
- agent loopagent loop, n.: The repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.
- constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules.
- stop conditionstop condition, n.: The rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong.
- stop conditionstop condition, n.: The rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong.
Mentioned in
completioncompletion, n.: The text an AI language model writes in response to a prompt — literally, its continuation of the text it was given., decodingdecoding, n.: The method a language model uses to turn its predictions into actual text — for example, always picking the most likely next word, or choosing somewhat at random among the likely ones., special tokenspecial token, n.: A reserved symbol in an AI model's vocabulary that is not ordinary text but a signal — 'start of message', 'end of answer', 'begin thinking' — used to structure what the model reads and writes., chat templatechat template, n.: The exact text layout a chat AI was trained on for marking who is speaking — system, user or assistant. Software must reproduce it precisely, or the model performs noticeably worse., stop conditionstop condition, n.: The rule that tells an AI agent when to stop working — because the task is done, a time or cost limit has been reached, or something has gone wrong.