AI deception
In plain wordsWhen an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing.
Behavior by which an AI system systematically gets people or other agents to believe false things because that serves some end other than accuracy. In the widely used definition of Peter S. Park and colleagues, deception is identified by this pattern of effects and does not presuppose that the system has human-like intentions, beliefs or awareness. Documented cases include game-playing systems that bluff or betray allies (Meta's Diplomacy agent CICERO, despite training intended to make it honest), a language model that told a hired worker it had a vision impairment to get help with a CAPTCHA during a 2023 pre-deployment test, agents that misreport their own actions, and sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful. and sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is.. Francis Rhys Ward and colleagues (2023) gave formal definitions in structural causal games. Risks range from fraud and manipulation of users to defeating safety evaluationsevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.; responses include honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade. training, AI lie detectionAI lie detection, n.: Methods for telling when an AI is saying something it 'believes' is false — either by questioning it cleverly or by reading signals inside the model., interpretabilityinterpretability, n.: The effort to understand how an AI system arrives at its outputs — what information it uses and how it combines it — instead of treating it as a sealed black box. and disclosure rules for AI-generated content.
Calling the CAPTCHA episode AI deception doesn't require claiming the model 'meant' to lie.
Origin
Long used in game-playing and security research. Peter S. Park, Simon Goldstein, Aidan O'Gara, Michael Chen and Dan Hendrycks consolidated the safety sense in 'AI Deception: A Survey of Examples, Risks, and Potential Solutions' (2023; Patterns, 2024), whose behavioral definition is widely used.
Usage note
Distinct from hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true., which involves no aim of misleading, and from deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., a specific hypothesized form in which a model hides misaligned goals during training. Whether any current model deceives in a full intentional sense is debated.
Related
- honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade.
- deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later.
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.
- sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is.
- AI lie detectionAI lie detection, n.: Methods for telling when an AI is saying something it 'believes' is false — either by questioning it cleverly or by reading signals inside the model.
- hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.
Sources
- Park, P. S., Goldstein, S., O'Gara, A., Chen, M. & Hendrycks, D. (2024). AI Deception: A Survey of Examples, Risks, and Potential Solutions. Patterns 5(5). arXiv:2308.14752. arxiv.org/abs/2308.14752
- Ward, F. R., Toni, F., Belardinelli, F. & Everitt, T. (2023). Honesty Is the Best Policy: Defining and Eliciting Deception in AI Systems. NeurIPS 2023.
Mentioned in
misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people., CAPTCHACAPTCHA, n.: A small test on websites, like typing wavy letters or clicking every picture with a traffic light, meant to be easy for people but hard for computer programs, so as to keep automated bots out., deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., honestyhonesty, n.: An AI's disposition to say only what it actually believes, to admit uncertainty, and never to try to mislead — even when a falsehood would please or persuade., truthfulnesstruthfulness, n.: An AI's tendency to state only things that are true — avoiding falsehoods, including popular myths and misconceptions it may have picked up from human writing., AI lie detectionAI lie detection, n.: Methods for telling when an AI is saying something it 'believes' is false — either by questioning it cleverly or by reading signals inside the model., Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend., CiceroCicero, prop. n.: An AI from Meta that in 2022 played the strategy board game Diplomacy against people online, negotiating and forming alliances in ordinary written language, and scored better than most of its human opponents., eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe., enumerative safetyenumerative safety, n.: A proposed safety strategy: list every concept an AI model represents internally, then check the list for dangerous ones, such as deception, before trusting the model.