Skip to content
AI360XpertBeta

Rectified Flow

Train a flow model, watch which noise becomes which image, then train again on those matched pairs. Each round straightens the paths, so sampling needs fewer steps.

Reflow replaces crossed noise-to-data lines with parallel straight ones, so one Euler step lands exactly on the target
Reflow replaces crossed noise-to-data lines with parallel straight ones, so one Euler step lands exactly on the target

Why Does This Exist?

Flow matching trains on straight segments, yet its samples still need several solver steps. The gap is pairing. Each training batch pairs random noise with random images, so the straight segments cross. Where two segments cross, one spot carries two true velocities, and the network learns their average instead of either one. The learned field curves, a curved field punishes big strides, and you pay for step after step at sampling time.

Rectified flow closes that gap with a second round of training called reflow. You simulate the trained model to see which noise actually becomes which image, then train again on those matched pairs. Matched pairs cross far less, the field straightens, and big strides stop hurting.

This page covers that second round: the crossing problem, the reflow loop, and when one step is enough. The straight line setup itself lives on the flow matching page. If the diffusion chain behind all of this is fuzzy, start with diffusion models. The endpoint-jump shortcut is a different answer to the same cost problem, covered in consistency models, and the stepping machinery is in diffusion samplers.

Think of It Like This

Paving paths where people already walk

First snowfall. A hundred people cross a field, each taking the straight line from their gate to their building, and their footprints cross everywhere. Now you pave. Not along the straight lines people intended, but along the trails they actually walked: one clean sidewalk per trail, laid parallel so the paths never intersect. Next morning everyone walks a paved path from gate to door in one steady stride.

Noise samples are the gates, images are the buildings, the first footprints are the base model's simulated trajectories, and paving is retraining on those matched pairs. The paved stride is the one-step sample.

Where it stops: paving never moves a building. If the base model sends a gate to the wrong building, reflow paves that wrong trail beautifully. Straighten only a model whose few-step samples already arrive at the right places.

How It Actually Works

When straight lines cross

Name the pieces. x0x_0 is one noise sample, drawn from a simple distribution p0p_0. x1x_1 is one data sample from the real distribution p1p_1. tt is a time in [0,1][0, 1], where t=0t = 0 means pure noise and t=1t = 1 means data. The point on the segment at time tt is xt=(1−t) x0+t x1x_t = (1 - t)\,x_0 + t\,x_1, and its velocity is the constant difference u=x1−x0u = x_1 - x_0. The network vθ(x,t)v_\theta(x, t), with weights θ\theta, takes a position xx and a time tt and tries to output that velocity. Training is squared error, L(θ)=E[ ∥vθ(xt,t)−(x1−x0)∥2 ]L(\theta) = E[\,\lVert v_\theta(x_t, t) - (x_1 - x_0)\rVert^2\,], averaged over random pairs and times.

Now watch two pairs collide. Pair A runs from x0=[0,1]x_0 = [0, 1] to x1=[2,−1]x_1 = [2, -1], so uA=[2−0, −1−1]=[2,−2]u_A = [2 - 0,\, -1 - 1] = [2, -2]. Pair B runs from x0=[0,−1]x_0 = [0, -1] to x1=[2,1]x_1 = [2, 1], so uB=[2,2]u_B = [2, 2]. At t=0.5t = 0.5, A stands at 0.5×[0,1]+0.5×[2,−1]=[1,0]0.5 \times [0, 1] + 0.5 \times [2, -1] = [1, 0], and B stands at 0.5×[0,−1]+0.5×[2,1]=[1,0]0.5 \times [0, -1] + 0.5 \times [2, 1] = [1, 0]. Same spot, same time, two different true arrows.

The loss at that spot is minimized by the average, [2,0][2, 0]. Check it: the gap to A is [0,2][0, 2], squares [0,4][0, 4], mean 2.02.0. The gap to B is [0,−2][0, -2], same 2.02.0. No single prediction can do better while both pairs share the spot, so the network files down both arrows into one stub. That stub is the blur you see in few-step samples.

Reflow, in two moves

First, simulate. Draw fresh noise, integrate the trained field from t=0t = 0 to t=1t = 1 with a careful solver, and record where each particle lands. In the toy, the averaged field [2,0][2, 0] carries [0,1][0, 1] forward one full stride: [0,1]+1×[2,0]=[2,1][0, 1] + 1 \times [2, 0] = [2, 1]. Notice the swap. It lands on B's image, not A's. That swap is the crossing made visible: the averaged field cannot tell the two travellers apart.

Second, retrain. Treat those simulated landings as the new dataset and run the same straight line training on them. The new pairs are ([0,1],[2,1])([0, 1], [2, 1]) with velocity [2,0][2, 0], and ([0,−1],[2,−1])([0, -1], [2, -1]) with velocity [2,0][2, 0]. Their midpoints are [1,1][1, 1] and [1,−1][1, -1]: two different spots, no shared input, nothing to average. The network can fit both arrows exactly, and the two segments run parallel instead of crossing.

You can repeat the loop, and each round pairs points the current field already connects, so crossings shrink and the field straightens further. One honest limit: reflow only redistributes pairings. It adds no capacity, so a small network still rounds the field off, and the endpoints stay exactly as right or wrong as the base model's.

Sampling in one stride

Euler integration on a straight segment is exact. From [0,1][0, 1] with velocity [2,0][2, 0] and step size h=1h = 1: [0,1]+[2,0]=[2,1][0, 1] + [2, 0] = [2, 1]. One network call, parked exactly on the target. That is the whole payoff.

Name the trade before you reach for it. You pay up front, simulating a dataset and training again, possibly more than once, to pay less per sample forever. Skip the second round when sampling is already cheap enough, as in a well tuned latent diffusion pipeline, or when you need exact likelihoods of your data, which belongs to flow-based models with their invertible layers.

Code

The point of this snippet is the crossing, replayed as arithmetic you can verify: two pairs share a midpoint, the learned average swaps their endpoints, and the reflowed pairs run parallel with exact one-step sampling.

import numpy as np

def velocity(x1: np.ndarray, x0: np.ndarray) -> np.ndarray:    return x1 - x0

def point_at(x0: np.ndarray, x1: np.ndarray, t: float) -> np.ndarray:    return (1.0 - t) * x0 + t * x1

def euler(x: np.ndarray, v: np.ndarray, h: float) -> np.ndarray:    return x + h * v

a0 = np.array([0.0, 1.0])a1 = np.array([2.0, -1.0])b0 = np.array([0.0, -1.0])b1 = np.array([2.0, 1.0])vA = velocity(a1, a0)  # pair A arrowvB = velocity(b1, b0)  # pair B arrowmidA = point_at(a0, a1, 0.5)  # A at t = 0.5midB = point_at(b0, b1, 0.5)  # B at t = 0.5avg = (vA + vB) / 2.0  # what the network learns at the shared spotloss = float(np.mean((avg - vA) ** 2))  # per-pair squared error, averagedswap = euler(a0, avg, 1.0)  # A under the averaged field, one full stridenew_mid = point_at(a0, swap, 0.5)  # reflowed pair midpoint at t = 0.5one = euler(a0, velocity(swap, a0), 1.0)  # one stride on the reflowed segmentprint(vA)  # -> [ 2. -2.]print(vB)  # -> [ 2.  2.]print(midA, midB)  # -> [1. 0.] [1. 0.]print(avg)  # -> [2. 0.]print(loss)  # -> 2.0print(swap)  # -> [2. 1.]print(new_mid)  # -> [1. 1.]print(one)  # -> [2. 1.]

Both midpoints land on [1,0][1, 0], which is why the network must average. The averaged stride from [0,1][0, 1] lands on [2,1][2, 1], B's image, proving the swap. After reflow the midpoint moves to [1,1][1, 1], clear of B's [1,−1][1, -1], and the single stride lands exactly.

Watch Out For

Reflow paves the wrong trail beautifully

Reflow keeps whatever pairing the base model produced, including its mistakes. If the base field sends some noise to the wrong image, the new dataset pairs them as ground truth and the next model learns that wrong pairing with full confidence.

The symptom is sharp, confident, wrong samples: one-step images that look clean but carry swapped attributes or mixed identities, and no amount of extra reflow rounds fixes them. Check few-step quality before you reflow, not after. If the base model's careful samples already go to the wrong places, fix the base model first. Reflow straightens paths; it does not correct destinations.

Cutting to one step while the field still curves

One exact stride needs a field that matches its segments. After a single reflow round on a hard dataset, yours likely does not, and jumping to one step brings the blur straight back.

The symptom is step-count dependent quality: smeared edges and washed out color at one step that clean up the moment you allow four or eight. Sweep the step count first and compare against a careful baseline. Prefer a midpoint or Heun step over plain Euler while the field is still curving, run another reflow round if the curve persists, and treat one step as earned, not default.

The Quick Version

  • Rectified flow is flow matching trained a second time on pairs the first model simulated itself, a loop called reflow.
  • Random pairing lets straight segments cross, and at each crossing the network learns the average arrow, which curves the field and forces small solver steps.
  • In the running example, pairs [0,1][0, 1] to [2,−1][2, -1] and [0,−1][0, -1] to [2,1][2, 1] share midpoint [1,0][1, 0], so the network learns [2,0][2, 0] with a per-pair loss of 2.02.0 neither arrow can beat.
  • Reflowed pairs ([0,1],[2,1])([0, 1], [2, 1]) and ([0,−1],[2,−1])([0, -1], [2, -1]) run parallel through distinct midpoints [1,1][1, 1] and [1,−1][1, -1], so one Euler stride of size 11 lands exactly.
  • You pay with an extra simulation plus retraining round to buy cheaper sampling forever, and it only pays off when sampling cost dominates.
  • Never reflow a model whose careful samples already land in the wrong places, since straightening preserves destinations, mistakes included.