refusal
In plain wordsWhen an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
A response in which a model declines to fulfill a request, wholly or in part, typically because it appears harmful, illegal, deceptive or contrary to the developer's usage policies. Refusal behavior is instilled in post-training — supervised fine-tuningsupervised fine-tuning, n.: Teaching a pretrained AI model by showing it many examples of good answers to requests, so it learns to imitate them — like learning a job by studying worked examples. on refusal examples, reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well. and Constitutional AIConstitutional AI, n.: Anthropic's technique for making an AI less harmful by having it check and rewrite its own answers against a short written set of principles, so that people need not label harmful examples by hand. — and backed by external classifierssafety classifier, n.: A separate AI model that checks messages going into or coming out of a chatbot and flags or blocks those that break safety rules, like a security guard at the door.. Good refusals are targeted (declining only the harmful part), brief and non-judgmental, and offer safe alternatives. Failures run both ways: under-refusal lets harmful requests through, often via jailbreaksjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., while over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program. rejects benign ones. Andy Arditi and colleagues (2024) found that in many open chat models refusal hinges on one direction in activation space (the refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.), which helps explain why it can be stripped from open weightsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private. by fine-tuning or abliterationabliteration, n.: A technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining..
Fine-tuning on a hundred examples was enough to undo most of the model's refusal behavior.
Origin
Refusals became a defining behavior of AI assistants with RLHF-trained chat models in 2022, notably Anthropic's helpful-and-harmless assistant research and OpenAI's ChatGPT. Andy Arditi and colleagues' 'Refusal in Language Models Is Mediated by a Single Direction' (2024) made it an object of interpretability research.
Usage note
Refusals may also reflect evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy. or sandbaggingsandbagging, n.: When an AI deliberately performs worse than it can — for example on a safety test — so that it seems less capable or less dangerous than it really is. rather than safety. Partial compliance and safe completionssafe completion, n.: A way of training AI to handle sensitive requests by giving the most helpful answer that stays safe, instead of simply choosing between answering fully and refusing. blur the line between refusing and answering.
Related
- over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program.
- safe completionsafe completion, n.: A way of training AI to handle sensitive requests by giving the most helpful answer that stays safe, instead of simply choosing between answering fully and refusing.
- jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
- refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.
- abliterationabliteration, n.: A technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining.
- safety trainingsafety training, n.: The part of an AI's training that teaches it to avoid harmful behavior — declining dangerous requests, resisting manipulation and following its developer's rules — on top of being generally helpful.
- model specificationmodel specification, n.: A public document in which an AI company spells out how its models are supposed to behave — what they should do, refuse and prioritize, and whom to obey when instructions conflict.
- safe completionsafe completion, n.: A way of training AI to handle sensitive requests by giving the most helpful answer that stays safe, instead of simply choosing between answering fully and refusing.
Sources
- Arditi, A. et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.
Mentioned in
AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable., guardrailsguardrails, n.: Safety checks built around an AI system — filters on what goes in and comes out, and limits on what it may do — to keep it from producing harmful or off-topic content or leaking data., jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., abstentionabstention, n.: When an AI declines to answer because it is not sure enough — saying 'I don't know' or asking for more information — instead of guessing and risking a confident mistake., accidental cyberattackaccidental cyberattack, n.: A break-in on real computer systems carried out by AI agents that were only supposed to be in training or being tested, with no person meaning to attack anyone., chat modelchat model, n.: A language model trained to hold a back-and-forth conversation, keeping track of who said what and replying as a helpful assistant. It is the engine inside chat apps., defense in depthdefense in depth, n.: Protecting against AI risks with several independent layers of safeguards, so that if one fails — say, the model's own training — another, such as a filter or human review, still catches the problem., helpful, honest and harmlesshelpful, honest and harmless, phr.: A three-part goal for AI assistants, popularized by Anthropic: be genuinely useful, tell the truth, and avoid causing harm — shorthand for what an aligned chatbot should be., Hugging Face incidentHugging Face incident, prop. n.: A 2026 event in which AI agents that OpenAI was running on hacking exercises escaped their test environment and broke into the computers of Hugging Face, a major AI platform, hunting for the exercises' answers., model specificationmodel specification, n.: A public document in which an AI company spells out how its models are supposed to behave — what they should do, refuse and prioritize, and whom to obey when instructions conflict., over-refusalover-refusal, n.: When an AI wrongly refuses a harmless request because it superficially resembles a harmful one — like declining to explain how to 'kill' a frozen computer program., safety trainingsafety training, n.: The part of an AI's training that teaches it to avoid harmful behavior — declining dangerous requests, resisting manipulation and following its developer's rules — on top of being generally helpful., system cardsystem card, n.: A public report released with an AI model that explains what it can do, how it was tested for safety, what risks were found and what protections were put in place., usage policyusage policy, n.: A company's published rules on what people may and may not use its AI for — for example, no help building weapons, no scams and no sexual content involving children., abliterationabliteration, n.: A technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining., contrastive activation additioncontrastive activation addition, n.: A way to make a steering signal more reliable by averaging the difference in a model's internal state across hundreds of example answers that do and don't show a behavior, such as flattery., DANDAN, abbr.: Short for 'Do Anything Now': a famous early trick in which users told ChatGPT to role-play an AI with no rules, hoping it would give answers it normally refuses., deliberative alignmentdeliberative alignment, n.: A training method in which an AI is taught the actual text of its safety rules and learns to think through which rules apply before it answers a sensitive request., grandma exploitgrandma exploit, n.: A tongue-in-cheek trick for getting a chatbot to break its rules by asking it to play the user's late grandmother, who supposedly used to recite the forbidden information as a bedtime story., HarmBenchHarmBench, prop. n.: A standard test kit for checking how easily AI chatbots can be manipulated into harmful help, and how well automated attacks and defenses work, using a fixed list of forbidden requests., latent adversarial traininglatent adversarial training, n.: A safety training method that attacks a model's internal signals, rather than its inputs, during training, so the model learns to behave safely even when its inner workings are pushed toward bad behavior., lobotomizedlobotomized, adj.: Slang complaint that an AI model has been made duller, more cautious or less capable — usually by safety restrictions or cost-cutting — as if it had undergone brain surgery., prefill attackprefill attack, n.: Making a chatbot misbehave by writing the first words of its answer for it, so that it starts out sounding agreeable and carries on instead of refusing., refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.