prompt tuning
In plain wordsTeaching a frozen language model a new task by learning a few made-up 'words' — lists of numbers rather than real text — that are placed before the input to steer the model.
A parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money. method that learns a small number of continuous embedding vectorsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. — a soft prompt, typically 5 to 100 positions long — prepended to the embedded input of a frozen language model and trained by backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction. on labeled examples. Its vectors need not correspond to any real word, and training can pack into them what a whole labeled dataset teaches; compared with prefix tuningprefix tuning, n.: Customizing a frozen language model by learning a short sequence of invisible 'virtual words' that is placed in front of every input at every layer, steering what the model produces., it touches only the input layer, which makes it simpler and cheaper. Results depend strongly on size: on T5 models with billions of parameters it reached the accuracy of tuning every weight and far exceeded GPT-3's few-shot prompting, while one frozen model can serve many tasks by switching prompts. With small models it underperforms other methods and can be unstable to train.
A 20-token soft prompt got the frozen 11B model within a point of full fine-tuning on our classification task.
Origin
Introduced by Brian Lester, Rami Al-Rfou and Noah Constant of Google in April 2021 (EMNLP 2021) as a simplification of prefix tuning; P-tuning (Xiao Liu and colleagues, March 2021) explored similar continuous prompts at about the same time.
Usage note
Not the same as prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it. or automatic prompt optimization, which search over readable text; the phrase is sometimes misused for manual prompt iteration. Distinct from instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text., which updates the model's weights.
Related
- prefix tuningprefix tuning, n.: Customizing a frozen language model by learning a short sequence of invisible 'virtual words' that is placed in front of every input at every layer, steering what the model produces.
- parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
- embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
- instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
- few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case.
- prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it.
- instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text.
Sources
- Lester, B., Al-Rfou, R. & Constant, N. (2021). The Power of Scale for Parameter-Efficient Prompt Tuning. EMNLP 2021. arXiv:2104.08691. arxiv.org/abs/2104.08691
Mentioned in
prompt engineeringprompt engineering, n.: The skill of writing requests to an AI — picking the words, examples, format and background information it receives — so that it reliably gives the result you want, without retraining it., instruction tuninginstruction tuning, n.: Training a language model on many tasks written as plain instructions with correct answers, so it learns to do what a request asks instead of just continuing the text., parameter-efficient fine-tuningparameter-efficient fine-tuning, n.: A family of tricks for customizing a large AI model by training only a tiny fraction of its numbers, or a small add-on, leaving the rest untouched to save memory and money., steeringsteering, n.: Guiding what an AI model does — its topic, tone or behavior — while it runs, by changing its instructions, its internal signals or how its next word is chosen, rather than by retraining it., context distillationcontext distillation, n.: Training a model to behave as if it had been given a long set of instructions, so it no longer needs to read those instructions every time., prefix tuningprefix tuning, n.: Customizing a frozen language model by learning a short sequence of invisible 'virtual words' that is placed in front of every input at every layer, steering what the model produces., prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand., textual inversiontextual inversion, n.: Teaching an image generator a new 'word' for a specific object or style from a few example pictures, leaving the model itself unchanged — only one vocabulary entry is added.