few-shot learning
In plain wordsGetting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.
Learning to perform a new task, or to recognize new classes, from only a few labeled examples, typically one to a few dozen per class; one-shot learning is the extreme of a single example. Benchmarks are framed as N-way K-shot episodes: classify queries into N novel classes given K labeled 'support' examples of each. Because a few examples cannot pin down a model on their own, few-shot methods rely on knowledge acquired beforehand: meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples. across many related tasks (prototypical networks, model-agnostic meta-learningmodel-agnostic meta-learning, n.: A training method that searches for a starting point for a model's internal settings from which it can learn any new task of a given kind with just a few small adjustments.), transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data. from pretrained representations followed by a light classifier or fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way., data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly. and Bayesian priors. With large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. the term gained a second, now dominant, meaning: performing a task from a few demonstrations placed in the prompt, with no weight updates at all (few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case., a form of in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.). Strong pretrained models made simple fine-tuning and prompting baselines competitive with specialized few-shot algorithms.
Given five labeled photos of a new defect type, the few-shot classifier reached 88% accuracy.
Origin
Li Fei-Fei, Rob Fergus and Pietro Perona's Bayesian one-shot learning of object categories (2003; journal version 2006) and Brenden Lake, Ruslan Salakhutdinov and Joshua Tenenbaum's 2015 Omniglot study drew attention to learning from very few examples. Matching networks (Oriol Vinyals and colleagues, 2016) and prototypical networks (Jake Snell and colleagues, 2017) defined the episodic benchmarks, and the 2020 GPT-3 paper 'Language Models are Few-Shot Learners' popularized the prompting sense.
Usage note
Two usages coexist: in classical machine learning the model is trained or adapted on the few examples; in LLM usage the examples are usually just supplied in the prompt. 'Shot' counts examples, not attempts.
Related
- meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples.
- few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case.
- in-context learningin-context learning, n.: A language model's ability to pick up a new task from instructions or examples placed in its input, on the spot, without any retraining — like a quick study learning a game's rules from one round.
- zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training.
- transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data.
- model-agnostic meta-learningmodel-agnostic meta-learning, n.: A training method that searches for a starting point for a model's internal settings from which it can learn any new task of a given kind with just a few small adjustments.
- data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly.
- zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training.
Sources
- Fei-Fei, L., Fergus, R. & Perona, P. (2006). One-Shot Learning of Object Categories. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(4).
- Snell, J., Swersky, K. & Zemel, R. (2017). Prototypical Networks for Few-shot Learning. Advances in Neural Information Processing Systems 30.
- Brown, T. B. et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33.
Mentioned in
transfer learningtransfer learning, n.: Reusing what a model learned on one task to get a head start on another — like a pianist picking up the organ faster than a beginner. It lets models learn new jobs from little data., few-shot promptingfew-shot prompting, n.: Showing an AI a handful of examples of a task — inputs with the answers you want — inside the prompt, so it can copy the pattern for a new case., GPT-3GPT-3, prop. n.: A huge 2020 OpenAI language model that could perform new tasks simply from instructions or a few examples written into its prompt, without retraining. It made prompting a new way of programming., meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples., task (machine learning)task (machine learning), n.: The job an AI model is meant to do, such as translating sentences, spotting spam or answering questions, defined by its inputs, its expected outputs and how success is scored., zero-shot learningzero-shot learning, n.: Doing a task without being shown a single example of it — say, an AI sorting reviews into positive and negative just from being told to, or recognizing an animal it never saw in training., case-based reasoningcase-based reasoning, n.: Handling a new situation by finding the most similar one dealt with before and adjusting what worked then, much as a cook adapts a remembered recipe to the ingredients at hand., DSPyDSPy, prop. n.: A free software framework in which programmers describe what each step of an AI pipeline should do, and the framework automatically works out good prompts and examples instead of hand-writing them., metric learningmetric learning, n.: Teaching a model to measure similarity: it learns to place items so that ones that should count as alike end up close together and different ones far apart., model-agnostic meta-learningmodel-agnostic meta-learning, n.: A training method that searches for a starting point for a model's internal settings from which it can learn any new task of a given kind with just a few small adjustments., Oriol VinyalsOriol Vinyals, prop. n.: A Spanish computer scientist who co-created key neural methods for translation and image captioning, led DeepMind's StarCraft-playing AlphaStar, co-led Google's Gemini models, and in 2026 co-founded the startup Discovery Loop., prompt optimizationprompt optimization, n.: Using software, often another AI model, to search automatically for the wording and examples that make an AI model perform best on a task, instead of tweaking prompts by hand., Siamese networkSiamese network, n.: A matching machine built from twin copies of one neural network: each copy turns one of two inputs, say two signatures or two face photos, into a summary, and the two summaries are compared.