activation steering
In plain wordsChanging an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model.
A family of inference-time techniques that control a neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. by intervening on its intermediate activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment. during the forward passforward pass, n.: The step where a neural network takes an input and runs it through its layers, one after another, to produce an output — its answer or prediction., leaving the weights unchanged. The commonest form adds a scaled steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it. v to the residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. at one or more layers and token positions (h → h + αv). Other forms project a direction out (directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does.), clamp sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. features (feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.), shift selected attention headattention head, n.: One of several parallel attention units inside a transformer layer. Each head learns its own way of deciding which words to connect, so different heads can specialize in different patterns. outputs (inference-time interventioninference-time intervention, n.: A technique that makes a language model answer more truthfully by slightly shifting the outputs of a few internal parts that seem to track truth, each time the model produces a word.) or apply learned low-rank edits (representation fine-tuningrepresentation fine-tuning, n.: A way of adapting a language model to a new task by training small, targeted edits to its internal signals instead of changing its weights, often with far fewer adjustable numbers than other methods.). Directions are obtained by contrasting activations on paired inputs (difference of means or principal components), from probeslinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly., by gradient-based optimization, or from interpretability features. Practitioners choose the layer (middle layers often work best), the strength α and the token positions. Typical failures are lost fluency at high strength, side effects on unrelated behavior and poor transfer across prompts.
Activation steering with a sycophancy vector reduced how often the model agreed with users' false claims.
Origin
Precursors include steering image generators through latent space (2019) and Sumanth Dathathri and colleagues' plug and play language modelplug and play language model, n.: An early way to steer what a language model writes — its topic or mood — by letting a small add-on classifier nudge the model's internal state at each word, without retraining the model. (2019), which nudged hidden states with classifier gradients. The additive form came from Nishant Subramani and colleagues (2022) and Alexander Matt Turner and colleagues (2023), who named the wider field 'activation engineering'; 'activation steering' became the prevailing term during 2023–2024.
Usage note
Steering is cheap and reversible but needs access to a model's internals, so outside the developer it is mainly practiced on open-weight modelsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private.. In the original AxBench evaluation (2025), prompting, then fine-tuning, beat every representation-steering method tested; later steering methods have narrowed the gap, but steering's main appeal still lies in precision, monitoring and research insight.
Related
- steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
- activation additionactivation addition, n.: A simple way to steer a language model: run two contrasting prompts such as 'Love' and 'Hate', take the difference between the model's internal states, and add it while the model writes.
- contrastive activation additioncontrastive activation addition, n.: A way to make a steering signal more reliable by averaging the difference in a model's internal state across hundreds of example answers that do and don't show a behavior, such as flattery.
- representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.
- feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.
- directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does.
- linear representation hypothesislinear representation hypothesis, n.: The idea that a neural network stores many concepts as simple directions inside its internal activity, so how much of a concept is present can be read off by measuring along its direction.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
Sources
- Turner, A. M. et al. (2023). Activation Addition: Steering Language Models Without Optimization. arXiv:2308.10248. arxiv.org/abs/2308.10248v4
- Subramani, N., Suresh, N. & Peters, M. (2022). Extracting Latent Steering Vectors from Pretrained Language Models. Findings of ACL 2022. aclanthology.org/2022.findings-acl.48
- Wu, Z. et al. (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025. arXiv:2501.17148. arxiv.org/abs/2501.17148
Mentioned in
sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful., activationactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment., activation vectoractivation vector, n.: The list of numbers a neural network produces at one layer for one input — a snapshot of its internal state. In steering, also a direction added to that state to shift behavior., steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining., steeringsteering, n.: Guiding what an AI model does — its topic, tone or behavior — while it runs, by changing its instructions, its internal signals or how its next word is chosen, rather than by retraining it., activation additionactivation addition, n.: A simple way to steer a language model: run two contrasting prompts such as 'Love' and 'Hate', take the difference between the model's internal states, and add it while the model writes., assistant axisassistant axis, n.: The main direction in a chatbot's internal activity that measures how much it is acting as its usual helpful-assistant self, as opposed to some other character it has slipped into., concept injectionconcept injection, n.: An experiment that plants a specific concept directly into an AI model's internal activity, then asks the model whether it notices anything unusual, to test whether its reports about itself are genuine., contrastive activation additioncontrastive activation addition, n.: A way to make a steering signal more reliable by averaging the difference in a model's internal state across hundreds of example answers that do and don't show a behavior, such as flattery., control vectorcontrol vector, n.: A steering signal saved as a small file that can be loaded alongside an open AI model to turn a trait — say, cheerfulness or laziness — up or down while it generates text., feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept., in-context vectorin-context vector, n.: A steering signal distilled from a few worked examples shown to a language model, so the model can follow the examples' pattern without having them repeated in every prompt., inference-time interventioninference-time intervention, n.: A technique that makes a language model answer more truthfully by slightly shifting the outputs of a few internal parts that seem to track truth, each time the model produces a word., latent adversarial traininglatent adversarial training, n.: A safety training method that attacks a model's internal signals, rather than its inputs, during training, so the model learns to behave safely even when its inner workings are pushed toward bad behavior., plug and play language modelplug and play language model, n.: An early way to steer what a language model writes — its topic or mood — by letting a small add-on classifier nudge the model's internal state at each word, without retraining the model., representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly., representation fine-tuningrepresentation fine-tuning, n.: A way of adapting a language model to a new task by training small, targeted edits to its internal signals instead of changing its weights, often with far fewer adjustable numbers than other methods., steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.