Skip to content
Last Dictionary
Plane
Interpretability & SteeringSecurity & MisuseSafety & AlignmentAI Engineering & MLOps
informalemerging

abliteration

noun/ab-lit-uh-RAY-shun/Also: abliterate, abliterated model, weight orthogonalization, refusal ablation

In plain wordsA technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining.

  1. The removal of a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.'s refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow. behavior by orthogonalizing its weights against the refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.. Once the unit direction r has been estimated from activation differences between harmful and harmless instructions, every matrix that writes into the residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. — the token embeddings and the attention and MLP output projections — is modified so that its outputs have no component along r. The result is equivalent to directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does. at every layer and position but is baked into an ordinary checkpoint that runs in standard inference software. 'Abliterated' versions of many open-weight modelsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private. circulate on model hubs. The edit usually costs some capability, which practitioners sometimes recover with brief further training such as direct preference optimizationdirect preference optimization, n.: A simpler way to teach an AI people's preferences: instead of building a separate scoring model, it trains directly on pairs of better and worse answers, making the better ones more likely..

    The abliterated checkpoint no longer refuses, but it scores a few points lower on reasoning benchmarks.

Wout′=(I−r^r^⊤)WoutW_{\text{out}}' = \left(I - \hat{r}\hat{r}^{\top}\right) W_{\text{out}}

Origin

Coined by the open-model developer known as FailSpy, whose model cards explain it as a play on 'ablation': 'ablate' plus 'obliterated', invented to set such models apart from 'uncensored' fine-tunes. FailSpy applied the weight-orthogonalization method of Arditi et al. (2024) to Llama 3 models and released the 'abliterator' library in May 2024; Maxime Labonne's tutorial 'Uncensor any LLM with abliteration' (June 2024) spread the word.

Usage note

Abliteration removes a safeguard rather than adding knowledge, and is a central example in debates over releasing model weights: safety training on open weights can be stripped cheaply. Distinguish it from a jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests., which manipulates inputs to a model that is left unchanged.

Related

See also
  • refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.
  • directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does.
  • refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow.
  • open-weight modelopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private.
  • jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.
  • machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
Contrast with
  • jailbreakjailbreak, n.: A trick for getting an AI chatbot to break its own rules — for example, to give dangerous instructions it would normally refuse — usually through cleverly worded requests.

Sources

  1. Labonne, M. (2024). Uncensor any LLM with abliteration. Blog post, June 2024. maximelabonne.substack.com/p/uncensor-any-llm-with-abliteration-d30148b7d43e
  2. Arditi, A. et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. arXiv:2406.11717. arxiv.org/abs/2406.11717

Mentioned in

refusalrefusal, n.: When an AI declines to do what it was asked — 'I can't help with that' — usually because the request seems harmful or breaks the rules it was trained to follow., steeringsteering, n.: Guiding what an AI model does — its topic, tone or behavior — while it runs, by changing its instructions, its internal signals or how its next word is chosen, rather than by retraining it., directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does., fine-tuning attackfine-tuning attack, n.: Removing an AI model's safety training by training it a little more on a small set of harmful examples — cheap to do when the model's files are public or a company offers custom training., prefill attackprefill attack, n.: Making a chatbot misbehave by writing the first words of its answer for it, so that it starts out sounding agreeable and carries on instead of refusing., refusal directionrefusal direction, n.: A single direction inside a chat model's internal activity that largely decides whether it refuses a request. Erasing it makes the model comply with almost anything; adding it makes the model refuse harmless requests.