Skip to content
Last Dictionary
Plane
Training & Post-trainingDeep LearningInterpretability & Steering

task arithmetic

nounAlso: task vector, task vectors

In plain wordsEditing an AI model by adding or subtracting the changes that fine-tuning made to it, as if skills were arrows that can be added together or reversed.

  1. A method of editing models in weight space using task vectors: the element-wise difference between a model's weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. after fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. on a task and the pretrained weights it started from. Three operations follow: adding a scaled task vector to the base model gives it the task; summing several builds one model with all of them (model mergingmodel merging, n.: Combining two or more trained AI models into one by blending their internal numbers directly, so the result shares their skills without any further training.); and negating a vector removes a behavior — toxic generation, say, or a task to be unlearned — while leaving others largely intact. Vectors can even be combined by analogy: for tasks related as 'A is to B as C is to D', the vectors of the first three yield gains on the fourth with no data from it. Task arithmetic works because fine-tuning moves weights only a short distance, in a roughly linear region around the pretrained model. Interference between task vectors limits how many can be combined, motivating methods such as TIES-merging and DARE. Related ideas include chat vectors, which transfer instruction following between models that share a base.

    Subtracting the toxicity task vector cut toxic completions sharply while the benchmark scores barely moved.

θnew=θpre+∑iλi τi,τi=θft(i)−θpre\theta_{\text{new}} = \theta_{\text{pre}} + \sum_{i} \lambda_i\, \tau_i, \qquad \tau_i = \theta^{(i)}_{\text{ft}} - \theta_{\text{pre}}

Origin

Introduced by Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi and Ali Farhadi in December 2022 and published at ICLR 2023. The name echoes the arithmetic of word vectors, as in the classic king − man + woman ≈ queen.

Usage note

Task vectors live in weight space; not to be confused with function vectorsfunction vector, n.: A compact internal signal, found inside a language model, that stands for a whole task — such as 'translate into French' or 'give the opposite word' — and can trigger that task when added to the model. or steering vectorssteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it., which are directions in activation space.

Related

See also
  • model mergingmodel merging, n.: Combining two or more trained AI models into one by blending their internal numbers directly, so the result shares their skills without any further training.
  • model soupmodel soup, n.: A single model made by averaging the internal numbers of several copies of a model that were fine-tuned in slightly different ways, often doing better than any one of them.
  • fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way.
  • low-rank adaptationlow-rank adaptation, n.: A cheap way to customize a big AI model: instead of changing all its billions of numbers, you train a small add-on of a few million numbers that slightly adjusts how it behaves.
  • machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
  • word2vecword2vec, prop. n.: A 2013 Google technique that learns a list of numbers for each word by training a very small network to predict which words appear near it across billions of words of text.
Contrast with
  • steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
  • function vectorfunction vector, n.: A compact internal signal, found inside a language model, that stands for a whole task — such as 'translate into French' or 'give the opposite word' — and can trigger that task when added to the model.

Sources

  1. Ilharco, G. et al. (2022). Editing Models with Task Arithmetic. ICLR 2023. arXiv:2212.04089. arxiv.org/abs/2212.04089

Mentioned in

model mergingmodel merging, n.: Combining two or more trained AI models into one by blending their internal numbers directly, so the result shares their skills without any further training., function vectorfunction vector, n.: A compact internal signal, found inside a language model, that stands for a whole task — such as 'translate into French' or 'give the opposite word' — and can trigger that task when added to the model., mode connectivitymode connectivity, n.: The discovery that two separately trained versions of a network, though their settings look very different, can usually be joined by a path along which the model keeps working well., model soupmodel soup, n.: A single model made by averaging the internal numbers of several copies of a model that were fine-tuned in slightly different ways, often doing better than any one of them.