AI lie detection
In plain wordsMethods for telling when an AI is saying something it 'believes' is false — either by questioning it cleverly or by reading signals inside the model.
Methods for detecting when a model asserts something its own internal representations indicate is false, as distinct from honest mistakes. Black-box detectors analyze behavior: Lorenzo Pacchiardi and colleagues' 2023 detector follows a suspected lie with a fixed battery of questions that have nothing to do with it and passes the model's yes-or-no replies to a simple classifier, a test that carried over to other models and settings. White-box detectors read activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment.: linear probeslinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly. trained to separate true from false statements, unsupervised methods such as contrast-consistent searchcontrast-consistent search, n.: A method for finding where a language model internally tracks whether statements are true, using only the logical rule that a statement and its opposite cannot both be true., and probes trained on instructed or strategic deception. Key difficulties are establishing what a model 'believes', distinguishing lying from role-play or fiction, and robustness: a detector used as a training signal may teach a model to evade it rather than to be honest. Lie detection supports alignment auditsalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss. and the eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe. agenda.
The deception probe fired on the agent's status report, which claimed tests had passed when they had not.
Origin
Early work includes Collin Burns and colleagues' unsupervised truth-finding (2022), Amos Azaria and Tom Mitchell's 'The Internal State of an LLM Knows When It's Lying' (2023) and Lorenzo Pacchiardi and colleagues' 'How to Catch an AI Liar' (2023). Nicholas Goldowsky-Dill and colleagues at Apollo Research (2025) applied probes to strategic deception.
Usage note
'Lie' presupposes something like belief; researchers operationalize it as a mismatch between a statement and the model's internal representation of truth, leaving philosophical questions open. See also AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade. and truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing..
Related
- AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing.
- eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe.
- contrast-consistent searchcontrast-consistent search, n.: A method for finding where a language model internally tracks whether statements are true, using only the logical rule that a statement and its opposite cannot both be true.
- linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly.
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
- alignment auditalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss.
- representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.
Sources
- Pacchiardi, L. et al. (2023). How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions. ICLR 2024.
- Azaria, A. & Mitchell, T. (2023). The Internal State of an LLM Knows When It's Lying. Findings of EMNLP 2023.
Mentioned in
AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade., sleeper agentsleeper agent, n.: An AI model secretly trained to behave normally until it sees a hidden trigger — such as a particular date — and then to behave badly, like a spy waiting to be activated., alignment auditalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss., contrast-consistent searchcontrast-consistent search, n.: A method for finding where a language model internally tracks whether statements are true, using only the logical rule that a statement and its opposite cannot both be true., eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe., mechanistic anomaly detectionmechanistic anomaly detection, n.: Flagging cases where an AI reaches its output by an unusual internal route — for instance because it is being tricked, or is tricking us — even when the output itself looks normal., representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.