steering
In plain wordsGuiding what an AI model does — its topic, tone or behavior — while it runs, by changing its instructions, its internal signals or how its next word is chosen, rather than by retraining it.
- 1
General
Any method for controlling a trained model's outputs toward a desired attribute — topic, style, sentiment, persona, safety behavior or task — at use time and without full retraining. Methods differ by where they intervene: in the input (promptingprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it., system promptssystem prompt, n.: Background instructions given to a chatbot before the conversation starts, setting its role, rules, tone and tools. Users usually never see it, but it shapes every reply., learned soft prompts as in prompt tuningprompt tuning, n.: Teaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model.); in the internal activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment. (activation steeringactivation steering, n.: Changing an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model.); or in the output distribution (logit biaslogit bias, n.: A setting that lets developers make specific words more or less likely in an AI model's output — even ban a word outright — by adjusting the model's scores before it chooses., constrained decodingconstrained decoding, n.: Forcing an AI model's output to follow a required format — such as valid code or a fill-in form — by blocking, at every step, any word that would break the rules., reweighting next-token probabilities with an attribute classifier). How readily a model follows such control is its steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining..
Prompting alone couldn't hold the persona for long conversations, so we switched to steering at layer 20.
- 2
Interpretability
Short for activation steeringactivation steering, n.: Changing an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model.: changing a model's behavior by modifying its internal activations during the forward passforward pass, n.: The step where a neural network takes an input and runs it through its layers, one after another, to produce an output — its answer or prediction., typically by adding a steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it. to the residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. or clamping an interpretable featurefeature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.. Because it is a causal intervention, steering also tests whether a representation found by probingprobing, n.: Testing what information is hidden inside an AI model by training a small, separate predictor on the model's internal activity: if the predictor can recover something, the model has stored it somewhere. actually drives behavior rather than merely correlating with it.
- 3
Generative media
Controlling an image, video or audio generator during sampling, chiefly through guidance — as in classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference., which extrapolates from an unconditional toward a conditional prediction of a diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static. — or by moving through a generator's latent space or editing its intermediate attention maps.
- 4
Agentsinformal
Redirecting a running AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. by sending it new instructions, corrections or priorities while it is still working on a task, so that it changes course at its next step without the user stopping the job and starting over. Coding agents and other agent harnessesharness, n.: The software around an AI model that gives it instructions and tools, carries out its actions and decides when it stops. The model is the engine; the harness is the rest of the car. increasingly accept such mid-task messages alongside queued follow-up requests.
Origin
Long used loosely for controlling generation; 'controllable text generation' systems around 2019 (CTRL, the plug and play language modelplug and play language model, n.: An early way to steer what a language model writes — its topic or mood — by letting a small add-on classifier nudge the model's internal state at each word, without retraining the model.) and Ali Jahanian, Lucy Chai and Phillip Isola's 'On the "steerability" of generative adversarial networks' (2019) framed the problem. The interpretability sense took hold with 'latent steering vectors' (Subramani et al., 2022) and activation addition (Turner et al., 2023).
Usage note
Distinguish steering, which changes behavior at use time and is reversible, from fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. and model editingmodel editing, n.: Changing one specific fact or behavior inside a trained AI model — for example, who leads a country — by directly adjusting a small part of it, without retraining the whole thing., which change the weights; folding a steering direction permanently into the weights, as in abliterationabliteration, n.: A technique for removing a chat model's ability to refuse requests by finding the internal direction responsible for refusals and permanently erasing it from the model's weights, with no retraining., blurs the line. The word is also used loosely for training-time shaping such as RLHFreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.; technical usage increasingly reserves it for inference-time control.
Related
- activation steeringactivation steering, n.: Changing an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model.
- steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
- steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining.
- feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.
- representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
- classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference.
- fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
- model editingmodel editing, n.: Changing one specific fact or behavior inside a trained AI model — for example, who leads a country — by directly adjusting a small part of it, without retraining the whole thing.
Mentioned in
Golden Gate ClaudeGolden Gate Claude, prop. n.: A one-day public demo in 2024 in which Anthropic turned up a single internal concept inside its Claude chatbot — the Golden Gate Bridge — so it steered almost every conversation back to the bridge., linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly., model editingmodel editing, n.: Changing one specific fact or behavior inside a trained AI model — for example, who leads a country — by directly adjusting a small part of it, without retraining the whole thing., steerabilitysteerability, n.: How easily an AI model's behavior — its tone, style, persona or rules — can be adjusted by the people using or deploying it, for example through instructions, without retraining., plug and play language modelplug and play language model, n.: An early way to steer what a language model writes — its topic or mood — by letting a small add-on classifier nudge the model's internal state at each word, without retraining the model.