PWC-Net Pyramid Flow Network

Solve big motion coarse, then fix details fine. PWC-Net builds flow as a pyramid of warps so small nets handle huge shifts.

Coarse flow warps features so fine levels only fix small residuals

Why Does This Exist?

FlowNet stacks full-resolution frames, which is heavy and weak on fine detail. PWC-Net exists to follow the classical pyramid instead: estimate coarse flow, warp features by it, then solve only the small residual at each finer level. The design cuts parameters drastically while sharpening boundaries.

Iteration without pyramids is RAFT; attention matching is GMFlow.

Think of It Like This

Folding a map before tracing streets

Find a city on a folded road map first, then unfold to trace its streets. Searching streets on the full unfolded map wastes time; the fold gives the neighborhood for free.

Coarse levels are the fold and fine levels trace streets. The analogy stops at warping: paper never stretches while feature maps warp by the current flow estimate.

How It Actually Works

A shared encoder builds feature pyramids for both frames. At level $l$ , the second-frame features warp by upsampled flow $2 \times \mathrm{up}(F^{l+1})$ , then a small cost volume correlates warped against first-frame features over a $\pm 4$ px window. A tiny decoder predicts the residual $\Delta F^l$ , and context networks refine the full-resolution output. Loss weights each level toward the finest, so coarse mistakes get corrected rather than frozen.

Worked numbers

A $64$ px shift is $2$ px at quarter resolution inside the $\pm 4$ window, then refines to $0.3$ px residual at full scale. Sintel final error lands near $2.0$ px with about $9$ million weights versus $150$ million for FlowNet2 stacks, roughly a seventeenfold saving.

Code

1def coarse_to_pixels(coarse: float, levels: int) -> float:2    return coarse * (2 ** levels)  # upsample coarse shift to full resolution3
4
5print(coarse_to_pixels(2.0, 5))  # -> 64.0

Watch Out For

Freezing coarse errors into fine levels

A wrong coarse vector warps features onto wrong content, and fine decoders polish the mistake. The symptom is large clean regions shifted as blocks. Keep the search window honest at coarse levels and supervise every scale during training.

The Quick Version

PWC-Net estimates flow coarse-to-fine with warping at each scale.
Cost volumes over $\pm 4$ px windows score matches locally.
Each level predicts only the residual above the upsampled coarse flow.
Error near $2.0$ px on Sintel final with far fewer weights than FlowNet.
A $64$ px shift becomes $2$ px at coarse scale.
Coarse mistakes propagate, so supervise all levels.