Skip to content
AI360XpertBeta

Flow Matching

Draw a straight line from random noise to a real picture, then teach a network the arrow pointing along that line. Sampling just follows the learned arrows home.

Flow matching trains one straight velocity arrow from noise to data, so two Euler half steps land exactly on the target.
Flow matching trains one straight velocity arrow from noise to data, so two Euler half steps land exactly on the target.

Why Does This Exist?

Diffusion models earn their pictures the hard way. They walk from pure noise to an image through many small cleanup steps along a curved, random path, and training one means learning that whole chain in reverse. It works, and the pictures speak for themselves. But every sample pays for all those steps, and the machinery around the schedule is heavier than the idea underneath.

Flow matching asks what happens if you throw the schedule away. Pair a noise sample with a data sample, draw the straight segment between them, and teach a network the velocity along that segment. If the learned arrows run straight, a solver can stride down them in a handful of steps instead of tiptoeing. That is the whole bet: straighter paths cost fewer steps.

The one idea you need first is the diffusion picture itself: a network that starts from static and repeatedly removes a little predicted noise until a face or a bedroom appears. Flow matching keeps the start (noise) and the end (data) and swaps the cleanup chain for a learned velocity field. Its closest siblings each take a different shortcut. Rectified flow straightens the paths further with a second round of training, consistency models jump straight to the endpoint, and score-based models learn arrows pointing toward high data density. This page is about the straight line version.

Think of It Like This

First morning at a big school

Picture a hundred new students scattered across the gym floor, each holding a card with an assigned seat in the bleachers. Painted arrows on the floor tell every student which way to walk, and each arrow follows one rule: the straight line from where that student stands to the seat on their card. The whistle blows, everyone walks their line at the same steady pace, and the scattered crowd becomes seated rows.

Each student is a noise sample, each seat is a data sample, each straight line is one noise to data segment, and the painted arrows are the network. Walking is sampling: you start somewhere random and follow the arrows until you arrive.

Where it stops: real students can see the whole gym and dodge each other. The network sees only one position and one moment, so when two straight lines cross it can paint just one arrow there, the average of the two. That averaging is where blur comes from, and fixing it is what the first pitfall below is about.

How It Actually Works

The straight line setup

Name the pieces. x0x_0 is one noise sample, drawn from a simple distribution p0p_0 (a standard Gaussian does the job). x1x_1 is one data sample from the real distribution p1p_1 (your images). tt is a time in [0,1][0, 1], where t=0t = 0 means pure noise and t=1t = 1 means data. The point on the segment at time tt is the plain interpolation:

xt=(1−t) x0+t x1x_t = (1 - t)\,x_0 + t\,x_1

Its velocity never changes along the segment. Differentiate with respect to tt and the tt terms collapse to the constant difference:

ut=x1−x0u_t = x_1 - x_0

So utu_t, the true velocity, is one fixed arrow per pair. The network vθ(x,t)v_\theta(x, t), with weights θ\theta, takes a position xx and a time tt and tries to output that arrow. You will sometimes hear this called conditional flow matching, because each training target is conditioned on one chosen pair.

The training loss

Training is regression, nothing fancier. Sample a random pair, a random time, build the point on the segment, and ask the network for the arrow:

L(θ)=E[ ∥vθ(xt,t)−(x1−x0)∥2 ]L(\theta) = E\left[\,\lVert v_\theta(x_t, t) - (x_1 - x_0)\rVert^2\,\right]

In words: the squared gap between the predicted arrow and the true straight line arrow, averaged over pairs and times. No ODE is solved while training, which is why people call it simulation free. One honest subtlety: where lines cross, the network sees the same (x,t)(x, t) with different true arrows and learns their average. That average still points the right way on balance, but it is softer than any single line, which is exactly why crossings cost you sharpness.

Worked example

Take noise x0=[−1,1]x_0 = [-1, 1] and data x1=[3,−1]x_1 = [3, -1]. The true velocity is:

u=x1−x0=[3−(−1), −1−1]=[4,−2]u = x_1 - x_0 = [3 - (-1),\ -1 - 1] = [4, -2]

At t=0.5t = 0.5 you stand halfway:

x0.5=0.5×[−1,1]+0.5×[3,−1]=[1,0]x_{0.5} = 0.5 \times [-1, 1] + 0.5 \times [3, -1] = [1, 0]

Say the network predicts [3.6,−1.8][3.6, -1.8] there. The error is [−0.4,0.2][-0.4, 0.2], the squared pieces are 0.160.16 and 0.040.04, and the mean over the two components is 0.100.10. That 0.100.10 is the loss on this pair and time.

Sampling follows the arrows

Sampling integrates what you taught. Start from fresh noise and step along the learned arrows from t=0t = 0 to t=1t = 1, which in symbols is dx/dt=vθ(x,t)dx/dt = v_\theta(x, t). With the true arrows from the example and two Euler steps of size h=0.5h = 0.5:

[−1,1]+0.5×[4,−2]=[1,0],[1,0]+0.5×[4,−2]=[3,−1][-1, 1] + 0.5 \times [4, -2] = [1, 0], \qquad [1, 0] + 0.5 \times [4, -2] = [3, -1]

Two strides land exactly on the data. That exactness holds because the velocity is constant on a straight segment. On a curved learned field it would not, and that gap is the price list for cutting steps: straighter field, bigger steps, fewer network calls.

That sets the trade clearly. You trade the diffusion schedule, many small well tested steps, for a learned field whose quality hinges on pairing and capacity. When crossings dominate, detail washes out. And here is the when not to use it: if you need exact likelihoods of your data, reach for flow-based models with their invertible layers instead, since this page's setup scores no likelihoods at all. If your latent diffusion pipeline already samples cheaply enough, the rewrite buys you little.

Code

The point of this snippet is the check, not the formula. It replays the worked example as arithmetic you can verify: the midpoint, the loss against a slightly wrong prediction, and the two Euler strides that land exactly.

import numpy as np

def velocity(x1: np.ndarray, x0: np.ndarray) -> np.ndarray:    return x1 - x0

def point_at(x0: np.ndarray, x1: np.ndarray, t: float) -> np.ndarray:    return (1.0 - t) * x0 + t * x1

def euler(x: np.ndarray, v: np.ndarray, h: float) -> np.ndarray:    return x + h * v

x0 = np.array([-1.0, 1.0])  # one noise samplex1 = np.array([3.0, -1.0])  # its paired data samplev = velocity(x1, x0)  # the straight line arrowmid = point_at(x0, x1, 0.5)  # halfway at t = 0.5pred = np.array([3.6, -1.8])  # a slightly wrong network outputloss = float(np.mean((pred - v) ** 2))  # squared error, averageds1 = euler(x0, v, 0.5)  # first half strides2 = euler(s1, v, 0.5)  # second half strideprint(v)  # -> [ 4. -2.]print(mid)  # -> [1. 0.]print(loss)  # -> 0.1print(s2)  # -> [ 3. -1.]

All four prints match the hand computation above: velocity [4,−2][4, -2], midpoint [1,0][1, 0], loss 0.100.10, and the second stride parked exactly on [3,−1][3, -1].

Watch Out For

Random pairs cross, and crossings blur

Independent pairing puts far apart endpoints on colliding lines. Around a crossing the network is asked for two different arrows at one spot, so it outputs the average, often a short stub pointing nowhere in particular. Your symptom is unmistakable: samples look washed out, and if you probe the field, velocities near the middle of the trip are much smaller than x1−x0x_1 - x_0 typically is.

Pair nearby points instead of random ones. In each minibatch, match noise samples to close data samples before drawing the segments (minibatch optimal transport), or train once, sample your own pairs from the model, and retrain on those straighter lines. That second pass is called reflow, and it has its own page: rectified flow.

Two steps work on paper, not on your first trained field

The worked example lands exactly because its arrows are perfectly straight. Your trained field is not, especially early on, so Euler error piles up with every stride. The symptom is drift: few step samples show smeared edges and off tone colors that clean up the moment you allow more steps.

Do not cut steps before the field earns it. Sweep the step count first, say 2, 5 and 10, and compare against a careful baseline. Prefer a midpoint or Heun step over plain Euler while the field is still curving, and only then distill or reflow down to the smallest count that holds quality.

The Quick Version

  • Flow matching trains a network to output the velocity along straight noise to data segments, and sampling integrates that velocity from t=0t = 0 to t=1t = 1.
  • The per pair target is just x1x_1 minus x0x_0, and training is plain squared error at random points on random segments with no ODE solved during training.
  • In the running example, noise [−1,1][-1, 1] to data [3,−1][3, -1] gives velocity [4,−2][4, -2] and midpoint [1,0][1, 0], and two Euler half steps land exactly on the data.
  • Random pairings let straight lines cross, and crossings teach averaged, blurry arrows, so pair nearby points or reflow to straighten them.
  • Skip it when you need exact likelihoods, where invertible flow-based models fit, and use it when sampler cost dominates and straighter paths buy back steps.