inverse scaling
In plain wordsWhen bigger or more heavily trained AI models do worse on a task than smaller ones — the opposite of the usual rule that scale brings improvement.
A pattern in which performance on a task worsens as models are scaled up in parameters, data or compute, contrary to the general improvement described by scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.. Analyzing the Inverse Scaling Prize tasks, Ian McKenzie and colleagues traced such reversals to four causes: misleading few-shot examples; easy distractor tasks that larger models latch onto instead of the real one; habits from training text that override instructions, such as reciting a familiar phrase instead of a requested variation; and faithful copying of undesirable patterns in the data, as in TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer., where larger models repeated more popular misconceptions. Some inverse-scaling trends reverse at still larger scales, producing U-shaped curves, and analogous effects appear with test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter., where longer reasoning lowers accuracy on certain tasks. Inverse scaling matters for safety because it shows that scale does not reliably remove — and can amplify — unwanted behaviors such as sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful..
This is clear inverse scaling: the 70B model follows the misleading examples more faithfully than the 7B one.
Origin
Stephanie Lin, Jacob Hilton and Owain Evans used the name 'inverse scaling' for the phenomenon in their TruthfulQA paper (September 2021). Ian McKenzie, Ethan Perez and colleagues popularized it with the public Inverse Scaling Prize in 2022, which rewarded submitted tasks showing the effect, and summarized the results in 'Inverse Scaling: When Bigger Isn't Better' (Transactions on Machine Learning Research, 2023).
Usage note
The mirror image of emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured., where performance jumps upward at scale; both show that small-model results can mislead about large-model behavior.
Related
- scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments.
- TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer.
- emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured.
- sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.
- test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter.
- Goodhart's lawGoodhart's law, n.: The rule of thumb that once a number is used as a target, people or AIs find ways to push it up that no longer reflect what it was meant to measure.
- overthinkingoverthinking, n.: When an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
Sources
- McKenzie, I. R., Lyzhov, A., Pieler, M. et al. (2023). Inverse Scaling: When Bigger Isn't Better. Transactions on Machine Learning Research.
Mentioned in
scaling lawsscaling laws, n.: Measured patterns showing that an AI model's error shrinks smoothly and predictably as the model is made bigger and given more data and computing power, so labs can forecast a giant model's performance from small, cheap experiments., emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured., TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer., overthinkingoverthinking, n.: When an AI reasoning model spends far more thinking than a problem needs — pages of deliberation for a simple sum — wasting time and money and sometimes talking itself out of a right answer., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious.