Skip to content
Last Dictionary
Plane
Generative MediaDeep LearningMathematics & Statistics

flow matching

nounAlso: conditional flow matching, CFM, flow-matching model, stochastic interpolants

In plain wordsA way to train generators that learns, for every point between random noise and real data, which direction to move, so that noise can be steered smoothly into new examples.

  1. A way of training a continuous normalizing flownormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is., which carries noise to data along a learned velocity field: a network v_θ(x, t) learns, for each point x and time t from 0 (noise) to 1 (data), which way and how fast probability should move. The velocity of the overall path is intractable, but simple per-example paths give a loss with the same gradient: commonly, each training step draws noise x₀ and data x₁, picks x_t = (1 − t)x₀ + t x₁ on the straight segment between them, and regresses the network onto that segment's velocity, x₁ − x₀. No differential equation is solved during training (the method is simulation-free); at generation time any ODE solver integrates the learned field from fresh noise. Diffusion corresponds to particular curved paths, so flow matching generalizes it; straighter paths are easier to learn and need fewer sampling steps. With classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference. and transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows. backbones, flow matching underlies Stable Diffusion 3, FLUXFLUX, prop. n.: A family of image-generating AI models from the German start-up Black Forest Labs, founded by researchers behind Stable Diffusion; some versions can be downloaded and run freely., Meta's Movie Gen and many audio and robotics models.

LCFM(θ)=Et,x0,x1∥vθ(xt,t)−(x1−x0)∥2,xt=(1−t) x0+t x1\mathcal{L}_{\text{CFM}}(\theta) = \mathbb{E}_{t, x_0, x_1}\big\lVert v_\theta(x_t, t) - (x_1 - x_0) \big\rVert^2, \quad x_t = (1-t)\,x_0 + t\,x_1

Origin

Introduced by Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel and Matt Le (Meta AI and the Weizmann Institute) in 'Flow Matching for Generative Modeling' (posted October 2022; ICLR 2023). Closely related formulations appeared independently the same autumn: rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps. (Xingchao Liu and colleagues) and stochastic interpolants (Michael Albergo and Eric Vanden-Eijnden).

Usage note

Time conventions vary: some papers put data at t = 0 and noise at t = 1. Flow matching models are often loosely called diffusion models.

Related

See also
  • rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps.
  • normalizing flownormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is.
  • diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
  • neural ordinary differential equationneural ordinary differential equation, n.: A neural network that, instead of passing data through a fixed stack of layers, describes how the data changes continuously over 'time' and uses an equation solver to follow that change.
  • diffusion transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows.
  • diffusion policydiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task.

Sources

  1. Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M. & Le, M. (2023). Flow Matching for Generative Modeling. ICLR 2023. arXiv:2210.02747. arxiv.org/abs/2210.02747

Mentioned in

diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static., text-to-image modeltext-to-image model, n.: An AI system that creates a picture from a written description, such as 'a fox reading a newspaper, in watercolor', usually within seconds., text-to-video modeltext-to-video model, n.: An AI that creates a short video clip — increasingly with matching sound — from a written description, such as 'a paper boat sailing down a rainy street'., classifier-free guidanceclassifier-free guidance, n.: A setting that makes an image generator follow the prompt more strictly by comparing what it would draw with and without the prompt, then exaggerating the difference., diffusion transformerdiffusion transformer, n.: A diffusion image or video generator whose core network is a transformer — the architecture behind chatbots — instead of the older image-specific design, which lets it improve more predictably as it grows., generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them., latent diffusion modellatent diffusion model, n.: A diffusion image generator that works on a shrunken, compressed version of each picture instead of the full-size pixels, which makes it far cheaper to train and run., music generationmusic generation, n.: Using AI to compose or produce music — from melodies written as notes to complete songs with vocals — often from a short text description like 'upbeat jazz with saxophone'., negative promptnegative prompt, n.: A second prompt listing what you do not want in a generated image — 'blurry, extra fingers, watermark' — which the generator then steers away from., random seedrandom seed, n.: A number that fixes the 'random' choices a computer makes; reuse the same seed and settings, and an image generator produces the same picture again., vision-language-action modelvision-language-action model, n.: An AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements., action chunkingaction chunking, n.: Having a robot's AI plan a short burst of upcoming movements, say the next second, all at once instead of one tiny motion at a time, which makes motion smoother and mistakes rarer., Black Forest LabsBlack Forest Labs, prop. n.: A German AI company founded in 2024 by original creators of Stable Diffusion, which makes the FLUX image-generation models., diffusion distillationdiffusion distillation, n.: Training a fast image generator to imitate a slow diffusion model, so that a picture that once took fifty steps can be made in one to four., diffusion policydiffusion policy, n.: A way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task., neural ordinary differential equationneural ordinary differential equation, n.: A neural network that, instead of passing data through a fixed stack of layers, describes how the data changes continuously over 'time' and uses an equation solver to follow that change., normalizing flownormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is., Physical IntelligencePhysical Intelligence, prop. n.: A San Francisco company, founded in 2024, that builds general-purpose AI 'brains' meant to control many different kinds of robots doing many different tasks., π0π0, prop. n.: A general-purpose AI model for controlling robots, released in 2024 by the start-up Physical Intelligence, that sees through cameras, follows typed or spoken instructions and moves robot arms deftly enough to fold laundry., rectified flowrectified flow, n.: A generator trained to move random noise toward real data along straight lines, which can then be straightened further so that a picture comes out in one or a few steps., score-based generative modelscore-based generative model, n.: A generator that learns, for any noisy example, which direction would make it look more like real data — then starts from pure noise and follows those directions step by step., Wasserstein distanceWasserstein distance, n.: A way of comparing two spreads of probability by imagining one as heaps of sand that must be shoveled into the shape of the other: the less shoveling, weighted by distance, the more alike they are.