instruction following
In plain wordsAn AI's ability to do exactly what it is asked, including details such as length, format, language and things to avoid, rather than something merely related.
The capability of a model to carry out tasks as specified in natural-language instructions, including explicit constraints on content, format, length, style and process, and to keep doing so across long contexts and multi-turn conversations. It is instilled mainly by instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text. and preference training, and evaluated with benchmarks of automatically checkable constraints (such as 'answer in fewer than 100 words' or 'use no commas'), with model-gradedLLM-as-a-judge, n.: Using one AI language model to grade or compare the answers of another, as a cheaper and faster stand-in for human reviewers. rubrics for complex requests, and with tests of whether models obey higher-priority instructions when instructions conflict (instruction hierarchyinstruction hierarchy, n.: A rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document.). Typical failures include dropping one of several constraints, drifting from instructions over long sessions, and over-literal compliance that misses the user's intent.
It writes well, but its instruction following is weak: it ignored the no-bullet-points rule twice.
Origin
Following natural-language instructions was studied in grounded settings such as navigation and game control from the 2000s. For language models it became a central goal with FLAN (2021) and OpenAI's InstructGPTInstructGPT, prop. n.: A 2022 OpenAI language model trained with human feedback to follow instructions; people preferred its answers to those of a model 100 times larger, and its recipe led directly to ChatGPT. paper, 'Training language models to follow instructions with human feedback' (Long Ouyang and colleagues, 2022); Jeffrey Zhou and colleagues' IFEval (2023) offered 25 types of verifiable instruction across about 500 prompts.
Related
- instruction-tuned modelinstruction-tuned model, n.: A language model given extra training on examples of instructions paired with good responses, so it does what you ask instead of merely continuing your text.
- instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
- instruction hierarchyinstruction hierarchy, n.: A rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document.
- steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining.
- reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.
- sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.
Sources
- Ouyang, L. et al. (2022). Training language models to follow instructions with human feedback. NeurIPS 2022. arxiv.org/abs/2203.02155
- Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D. & Hou, L. (2023). Instruction-Following Evaluation for Large Language Models. arXiv:2311.07911. arxiv.org/abs/2311.07911
Mentioned in
post-trainingpost-training, n.: Everything done to an AI model after its initial bulk training on huge amounts of text: the extra teaching that turns a raw text predictor into a helpful, safe assistant or a careful reasoner., instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text., instruction-tuned modelinstruction-tuned model, n.: A language model given extra training on examples of instructions paired with good responses, so it does what you ask instead of merely continuing your text., steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining., IFEvalIFEval, prop. n.: A test of whether AI models obey precise formatting and content rules — such as 'use exactly three bullet points' or 'answer in capital letters' — that a computer program can check automatically., instruction hierarchyinstruction hierarchy, n.: A rule built into AI assistants about whose instructions win when they conflict: the developer's beat the user's, and the user's beat any instructions hidden in a web page or document., MT-BenchMT-Bench, prop. n.: A set of 80 two-part questions — in writing, reasoning, math, coding and more — whose answers are graded by a strong AI model, used to compare chat assistants quickly.