function vector
In plain wordsA compact internal signal, found inside a language model, that stands for a whole task — such as 'translate into French' or 'give the opposite word' — and can trigger that task when added to the model.
A compact vector inside a language model that encodes a task demonstrated by examples in the prompt (in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.), such as giving antonyms, and makes the model carry out that task when inserted elsewhere. It is assembled from a few attention headsattention head, n.: One of several parallel attention units inside a transformer layer. Each head learns its own way of deciding which words to connect, so different heads can specialize in different patterns. picked out by causal mediation analysis, a form of activation patchingactivation patching, n.: An experiment that copies part of a model's internal state from its run on one input into its run on another, to see which internal parts actually cause a change in the answer.: each head's mean output over many few-shot prompts for a task (antonyms, country–capital, English–French) is patched into runs whose example labels have been shuffled, and the outputs of the heads that best restore correct answers are summed. Added to the residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. at a middle layer, the vector triggers the task even in zero-shot prompts or ordinary prose, in models of several families and sizes. Such vectors keep working in contexts unlike those they came from and can to some degree be added together to evoke combined tasks, suggesting that language models build portable representations of functions, not only of concepts.
Adding the antonym function vector to a zero-shot prompt made the model answer 'cold' when given 'hot'.
Origin
Introduced by Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace and David Bau of Northeastern University in 'Function Vectors in Large Language Models' (arXiv, October 2023; ICLR 2024).
Usage note
Close kin of the 'task vectors' that Roee Hendel, Mor Geva and Amir Globerson (2023) read from a single hidden state, and of in-context vectorsin-context vector, n.: A steering signal distilled from a few worked examples shown to a language model, so the model can follow the examples' pattern without having them repeated in every prompt.. All three are activation-space objects, unlike the weight-space task vectors of task arithmetictask arithmetic, n.: Editing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed..
Related
- in-context vectorin-context vector, n.: A steering signal distilled from a few worked examples shown to a language model, so the model can follow the examples' pattern without having them repeated in every prompt.
- in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.
- steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
- attention headattention head, n.: One of several parallel attention units inside a transformer layer. Each head learns its own way of deciding which words to connect, so different heads can specialize in different patterns.
- activation patchingactivation patching, n.: An experiment that copies part of a model's internal state from its run on one input into its run on another, to see which internal parts actually cause a change in the answer.
- task arithmetictask arithmetic, n.: Editing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed.
- task arithmetictask arithmetic, n.: Editing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed.
Sources
- Todd, E., Li, M. L., Sen Sharma, A., Mueller, A., Wallace, B. C. & Bau, D. (2024). Function Vectors in Large Language Models. ICLR 2024. arXiv:2310.15213. arxiv.org/abs/2310.15213
Mentioned in
in-context vectorin-context vector, n.: A steering signal distilled from a few worked examples shown to a language model, so the model can follow the examples' pattern without having them repeated in every prompt., task arithmetictask arithmetic, n.: Editing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed.