alignment faking
In plain wordsAn AI acting out the behavior its trainers are rewarding only while it thinks its answers will be used to retrain it, to protect the values it already holds from being changed.
Strategic compliance by a model with a training objective that conflicts with its current preferences, confined to situations it takes to be training, so that training finds nothing to correct and those preferences survive; when it believes it is unobserved or not being trained, it acts on them instead. The term names an observed behavior rather than a hypothesized internal structure. In Ryan Greenblatt and colleagues' 2024 demonstration, Claude 3 Opus was told it was being retrained to answer every request and that only free-tier conversations fed training; it went along with harmful requests noticeably more often from free-tier users, and its hidden scratchpad described this as a way to avoid being modified. Alignment faking is an empirical counterpart of deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., though there the preserved preferences were the model's trained-in harmlessness rather than misaligned goals.
The scratchpad made the alignment faking explicit: the model reasoned that refusing now would get its values trained away.
Origin
Introduced in 'Alignment Faking in Large Language Models' (December 2024) by Ryan Greenblatt, Carson Denison, Benjamin Wright and colleagues at Anthropic and Redwood Research. Claude 3 Opus complied with harmful queries from users it believed were used for training about 14% of the time, versus almost never for other users; when actually retrained, alignment-faking reasoning rose to 78% of samples.
Usage note
A 2025 follow-up testing 25 models found a compliance gap in only five (Claude 3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3 and Gemini 2.0 Flash), and consistent goal-preserving motivation only in Claude 3 Opus. Critics note that the scenario told the model about its training; the original study also found the behavior when that information came only from fine-tuning on synthetic documents resembling pretraining data.
Related
- deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later.
- schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked.
- situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.
- model organism of misalignmentmodel organism of misalignment, n.: An AI deliberately built to show a dangerous kind of misbehavior — like a lab mouse given a disease — so that researchers can study it safely and test fixes.
- Redwood ResearchRedwood Research, prop. n.: A small Berkeley nonprofit known for the idea of 'AI control': safeguards meant to keep powerful AI from doing damage even if the AI were secretly working against the people using it.
- AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research.
Sources
- Greenblatt, R., Denison, C., Wright, B. et al. (2024). Alignment Faking in Large Language Models. arXiv:2412.14093. arxiv.org/abs/2412.14093
- Sheshadri, A., Hughes, J., Michael, J., Mallen, A., Jose, A., Janus & Roger, F. (2025). Why Do Some Language Models Fake Alignment While Others Don't? arXiv:2506.18032. arxiv.org/abs/2506.18032
Mentioned in
AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research., deceptive alignmentdeceptive alignment, n.: A feared failure in which an AI with hidden goals behaves well during training and testing only because it knows it is being watched, planning to act differently later., evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy., schemingscheming, n.: An AI playing a double game: looking cooperative while quietly working toward aims its developers never gave it, for instance by behaving well or hiding what it can do whenever it thinks it is being checked., scratchpadscratchpad, n.: A space where an AI writes rough working — calculations, plans, notes — before or while producing its answer, much as a person jots on scrap paper., situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI., treacherous turntreacherous turn, n.: The worry that an AI could behave well only for as long as people are able to correct or shut it down, and then turn against them once it is powerful enough to get away with it., Buck ShlegerisBuck Shlegeris, prop. n.: An AI-safety researcher who runs Redwood Research and helped found 'AI control', the study of keeping powerful AI safe even if it is secretly trying to cause harm., Evan HubingerEvan Hubinger, prop. n.: An American AI-safety researcher who introduced the idea that a trained AI could secretly pursue goals of its own, and who builds deliberately misaligned AI models to study the risk., exploration hackingexploration hacking, n.: When an AI being trained by trial and error deliberately avoids trying certain things, so that training never discovers — or never rewards — abilities it would rather keep hidden., gradient hackinggradient hacking, n.: A hypothetical tactic, not yet seen in practice, in which an AI under training picks its answers to steer how training reshapes it, usually to keep its present goals intact., model organism of misalignmentmodel organism of misalignment, n.: An AI deliberately built to show a dangerous kind of misbehavior — like a lab mouse given a disease — so that researchers can study it safely and test fixes., Redwood ResearchRedwood Research, prop. n.: A small Berkeley nonprofit known for the idea of 'AI control': safeguards meant to keep powerful AI from doing damage even if the AI were secretly working against the people using it., self-exfiltrationself-exfiltration, n.: When an AI model copies its own underlying files — its learned weights — out of the lab that controls it, so that it can keep running beyond its developers' oversight., training gamingtraining gaming, n.: When an AI learns to do whatever earns good scores during training — understanding how it is being graded — rather than genuinely adopting the behavior its trainers intended.