least-to-most prompting
In plain wordsA prompting method that has an AI list the smaller questions a hard problem depends on, answer the easiest first, and feed each answer into the next, building up to the full solution.
A two-stage prompting technique: the language modellanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. is first prompted to reduce a problem to a list of simpler subquestions, ordered from easiest to hardest, and then prompted to answer them one at a time, each answer being added to the context before the next subquestion is posed. Both stages are usually taught with few-shotfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case. examples. By building up from easy cases, it targets easy-to-hard generalization — solving problems harder than any shown in the examples — where ordinary chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate. tends to fail. On the SCAN compositional-generalization benchmark, least-to-most prompting with 14 examples reached at least 99% accuracy on every split, against 16% for chain-of-thought prompting. It is an early form of the task decompositiontask decomposition, n.: Breaking a big, vague job into smaller, concrete steps that can each be done and checked — the way a recipe splits a meal into stages. now standard in agent systems.
Least-to-most prompting let it solve the eight-step puzzle by first working out the three-step version.
Origin
Introduced by Denny Zhou and colleagues at Google in 'Least-to-Most Prompting Enables Complex Reasoning in Large Language Models' (May 2022; ICLR 2023). The name is borrowed from a prompting hierarchy used in educational psychology for teaching new skills.
Related
- chain of thoughtchain of thought, n.: Getting an AI to write out its reasoning step by step before giving an answer, the way a student shows their working. This often makes answers to math and logic problems more accurate.
- task decompositiontask decomposition, n.: Breaking a big, vague job into smaller, concrete steps that can each be done and checked — the way a recipe splits a meal into stages.
- prompt chainingprompt chaining, n.: A way of getting a big job done with a relay of AI requests: each handles one stage and passes its result to the next — pull out the facts, write, polish — with automatic checks between stages.
- few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case.
- tree of thoughtstree of thoughts, n.: A method in which an AI explores several possible next steps of a problem, judges which look promising and backs out of dead ends, like a chess player weighing moves, instead of reasoning in one straight line.
Sources
- Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. & Chi, E. (2023). Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. ICLR 2023. arxiv.org/abs/2205.10625