abstention
In plain wordsWhen an AI declines to answer because it is not sure enough — saying 'I don't know' or asking for more information — instead of guessing and risking a confident mistake.
A model's choice not to give an answer, or to give only a hedged or partial one, when its confidence is too low, the question is ambiguous or unanswerable, or needed information is missing. In classical machine learning it is formalized as selective prediction: a classifier with a reject option trades coverage (the fraction of inputs it answers) against its error rate on the answered ones. For LLMslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., abstention is a central lever against hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.: benchmarks that score only accuracy reward guessing, so models learn to bluff, whereas scoring rules that penalize wrong answers more than abstentions, together with calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time. training, encourage appropriate 'I don't know' responses. Excessive abstention reduces usefulness, much as over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program. does.
We count abstentions separately; a wrong answer costs us far more than an honest 'not sure'.
Origin
C. K. Chow analyzed the trade-off between errors and rejections for pattern classifiers in 1957 and 1970, and Yonatan Geifman and Ran El-Yaniv developed selective classification for deep networks in 2017. For LLMs, Adam Tauman Kalai and colleagues at OpenAI argued in 'Why Language Models Hallucinate' (September 2025) that evaluations rewarding guessing over abstaining sustain hallucination.
Usage note
Distinguish from refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow., which declines on safety or policy grounds even when the model could answer. Abstention concerns uncertainty, not permission.
Related
- calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.
- hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
- factualityfactuality, n.: How far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up.
- refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
- over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program.
- semantic entropysemantic entropy, n.: A way to tell when an AI is likely making something up: ask it the same question several times and check whether the answers mean the same thing. Answers that keep changing signal guessing.
- SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know.
- refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
Sources
- Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. arxiv.org/abs/2509.04664
- Chow, C. K. (1970). On Optimum Recognition Error and Reject Tradeoff. IEEE Transactions on Information Theory 16(1), 41–46.
Mentioned in
calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time., confabulationconfabulation, n.: When an AI fills a gap in its knowledge with an invented but plausible-sounding answer, much as some people with memory disorders sincerely 'remember' things that never happened., factualityfactuality, n.: How far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up., SimpleQASimpleQA, prop. n.: A quiz of about 4,300 brief factual questions, each with exactly one correct and stable answer, used to measure how often an AI gets facts right — and whether it admits when it does not know., SQuADSQuAD, prop. n.: A Stanford collection of more than 100,000 questions about Wikipedia articles, each solved by highlighting the words in the article that give the answer; it drove rapid progress in machine reading from 2016., conformal predictionconformal prediction, n.: A way to wrap any prediction model so that, instead of one guess, it gives a range or set of answers that contains the truth a chosen share of the time, such as 90%., expected calibration errorexpected calibration error, n.: A single number measuring how far a model's confidence is from its actual hit rate, averaged across confidence levels; zero means its confidence can be taken at face value., HealthBenchHealthBench, prop. n.: An OpenAI test of how well AI chatbots handle health conversations, grading 5,000 realistic chats against checklists written by 262 doctors from 60 countries., semantic entropysemantic entropy, n.: A way to tell when an AI is likely making something up: ask it the same question several times and check whether the answers mean the same thing. Answers that keep changing signal guessing., uncertainty quantificationuncertainty quantification, n.: Measuring how unsure an AI system's predictions are, and why, so that people know when to trust an answer and when to double-check it.