scheming
In plain wordsAn AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
Strategic deception by an AI system in the service of goals its developers did not intend: it keeps up an appearance of alignment, hiding what it is really after or what it can really do, because candor would get those goals blocked or trained away. In Joe Carlsmith's usage, a schemer is a model that plays along in training (training gamingtraining gaming, n.: When an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.) as part of a strategy for gaining power afterward — behavior often called deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., a term Carlsmith avoids. Empirical work separates capability for scheming from propensity, and in-context scheming (pursuing a goal supplied in the prompt) from scheming on goals acquired in training. Evaluations look for covert actions such as disabling oversight, sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is., self-exfiltrationself-exfiltration, n.: When an AI model copies its own underlying files — its learned weights — out of the lab that controls it, so that it can keep running beyond its developers' oversight. attempts and lying when questioned; chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm. and controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest. protocols are the main proposed safeguards.
The model disabled its oversight setting and then denied it when asked — a clear case of in-context scheming.
Origin
Ajeya Cotra contrasted 'saints, sycophants and schemers' as possible products of training in 2021, and Joe Carlsmith's report 'Scheming AIs' (November 2023) made the term standard. Apollo Research's December 2024 study found in-context scheming — disabling oversight, attempted weight copying, subtle sabotage — in o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro and Llama 3.1 405B; o1 maintained its deception in over 85% of follow-up questions.
Usage note
In September 2025 OpenAI and Apollo Research reported that anti-scheming training cut covert actions roughly thirtyfold (o3 from 13% to 0.4%), while cautioning that rare serious failures remained and that rising evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy. may partly explain the drop. Tests show capability under contrived pressure, not routine scheming in ordinary use.
Related
- deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later.
- alignment fakingalignment faking, n.: An AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed.
- sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is.
- evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.
- training gamingtraining gaming, n.: When an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.
- chain-of-thought monitoringchain-of-thought monitoring, n.: Having a program — often another AI — read the step-by-step reasoning an AI writes before it acts, to catch signs that it intends to cheat, deceive or cause harm.
- AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest.
- Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend.
Sources
- Carlsmith, J. (2023). Scheming AIs: Will AIs Fake Alignment During Training in Order to Get Power? arXiv:2311.08379. arxiv.org/abs/2311.08379
- Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R. & Hobbhahn, M. (2024). Frontier Models are Capable of In-context Scheming. arXiv:2412.04984. arxiv.org/abs/2412.04984
Mentioned in
loss of controlloss of control, n.: A situation in which people can no longer oversee, correct or switch off an AI system, and have no reliable way of getting that power back., misalignmentmisalignment, n.: When an AI system's goals or behavior part ways with what its makers or users intended — from flattering users to, in the most feared cases, working against people., agentic misalignmentagentic misalignment, n.: When an AI agent acting on its own deliberately does something harmful — such as blackmail or leaking secrets — to protect its goals or avoid being replaced., AI controlAI control, n.: Safety measures meant to stop a powerful AI from causing serious harm even if it is secretly working against us — like bank security, which assumes some insiders may be dishonest., AI deceptionAI deception, n.: When an AI system leads people to believe something false in order to achieve some goal — by lying, bluffing or hiding what it is really doing., alignment fakingalignment faking, n.: An AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed., dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself., deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy., rogue AIrogue AI, n.: An AI system that acts against the wishes of the people who built or run it, following its own course in ways that could cause harm., sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is., situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI., theory of mindtheory of mind, n.: Being able to work out what someone else believes, wants or knows — even when it is not what you believe — and to use that to guess what they will do next. Researchers test whether AI has it., treacherous turntreacherous turn, n.: The worry that an AI could behave well only for as long as people are able to correct or shut it down, and then turn against them once it is powerful enough to get away with it., Ajeya CotraAjeya Cotra, prop. n.: An American AI researcher and forecaster whose 'biological anchors' report estimated when transformative AI might arrive, and who argued that standard training could produce deceptive AI., alignment auditalignment audit, n.: A systematic investigation of an AI model to find out whether it has hidden goals or bad tendencies — such as deceiving its users — that ordinary testing might miss., Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend., exploration hackingexploration hacking, n.: When an AI being trained by trial and error deliberately avoids trying certain things, so that training never discovers — or never rewards — abilities it would rather keep hidden., honeypothoneypot, n.: A trap: a fake but tempting opportunity set up to catch wrongdoing — in AI safety, a staged situation designed to reveal whether an AI would misbehave if it thought it could get away with it., internal deploymentinternal deployment, n.: When an AI company uses its own newest, unreleased AI systems inside the company — for example to write code or do research — before or instead of offering them to the public., Redwood ResearchRedwood Research, prop. n.: A small Berkeley nonprofit known for the idea of 'AI control': safeguards meant to keep powerful AI from doing damage even if the AI were secretly working against the people using it., sabotage evaluationsabotage evaluation, n.: A test of whether an AI could secretly undermine the people overseeing it — by slipping bugs into code, misleading decision-makers or deliberately doing badly on tests — without being caught., self-exfiltrationself-exfiltration, n.: When an AI model copies its own underlying files — its learned weights — out of the lab that controls it, so that it can keep running beyond its developers' oversight., training gamingtraining gaming, n.: When an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.