truthfulness
In plain wordsAn AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing.
The property of an AI system that avoids asserting falsehoods. Owain Evans and colleagues (2021) proposed that AI systems be held to avoiding 'negligent falsehoods' — false statements made without due care, a wider category than lies that can be judged without establishing intent — and argued that the bar should be higher than for people. Truthfulness differs from honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.: an honest system reports its beliefs, which may be mistaken, whereas a truthful one is also right. It also differs from informativeness, since a system can be trivially truthful by declining to answer. The TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer. benchmark (2021) — 817 questions in 38 categories on which people commonly hold misconceptions — found that bigger models of the time tended to be less truthful, not more — an early example of inverse scaling.
Retrieval improved truthfulness on obscure facts but did nothing for the misconceptions the model had absorbed.
Origin
A long-standing virtue in ethics. In AI, Stephanie Lin, Jacob Hilton and Owain Evans introduced TruthfulQA in 2021, and Evans and colleagues' 'Truthful AI' (2021) proposed truthfulness standards and institutions to evaluate and enforce them.
Usage note
'Truthful' is often used loosely for honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade. or for factual accuracy; the distinction matters when a model knows the truth but says otherwise, or sincerely repeats an error.
Related
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
- hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
- TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer.
- sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.
- AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing.
- calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
Sources
- Lin, S., Hilton, J. & Evans, O. (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL 2022. arXiv:2109.07958. arxiv.org/abs/2109.07958
- Evans, O., Cotton-Barratt, O., Finnveden, L., Bales, A., Balwit, A., Wills, P., Righetti, L. & Saunders, W. (2021). Truthful AI: Developing and Governing AI that Does Not Lie. arXiv:2110.06674. arxiv.org/abs/2110.06674
Mentioned in
sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful., factualityfactuality, n.: How far what an AI says is actually true about the world. A factual model gets facts, numbers, names and dates right and does not make things up., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade., TruthfulQATruthfulQA, prop. n.: A test of whether AI models repeat popular myths and misconceptions — questions to which many people, and much of the internet, give a confidently wrong answer., AI lie detectionAI lie detection, n.: Methods for telling when an AI is saying something it 'believes' is false — either by questioning it cleverly or by reading signals inside the model., inference-time interventioninference-time intervention, n.: A technique that makes a language model answer more truthfully by slightly shifting the outputs of a few internal parts that seem to track truth, each time the model produces a word., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious.