Multi-Plane Images for 3D Views

Multi-plane images stack front-parallel RGBA layers so new views render by warping and blending, with no mesh needed.

Why Does This Exist?

Depth maps store one layer and tear at occlusions. Full volumes cost too much for phones. Multi-plane images (MPI) keep $D = 32$ front-parallel RGBA planes at fixed depths, which render nearby views by warping each plane and alpha-blending back to front. Novel view synthesis uses MPI for fast small-baseline results.

Think of It Like This

Glass panes of a theater diorama

A stage scene is painted across 32 glass panes stacked in depth. Sliding your head sideways reveals parallax because near panes shift more. Walk too far and you see pane edges and gaps.

Planes are panes and alpha is paint coverage. The analogy stops at discreteness: real depth is continuous while panes are fixed.

How It Actually Works

Plane $d$ holds color $C_d$ and alpha $\alpha_d$ at depth $z_d$ . Rendering warps each plane to the new view by homography $H_d$ from camera motion, then composites back to front: $C = \sum_d T_d \alpha_d C_d$ with transmittance $T_d = \prod_{j>d}(1-\alpha_j)$ . Depths space inversely between $2$ m and $60$ m for near precision.

Worked example

Two planes at $5$ m and $10$ m see a pole. Near plane has $\alpha = 0.9$ , $C = [0.2, 0.2, 0.2]$ ; far has $\alpha = 1.0$ , $C = [0.8, 0.8, 0.8]$ . Back-to-front gives $C = 0.9 \times [0.2, 0.2, 0.2] + (1-0.9) \times [0.8, 0.8, 0.8] = [0.26, 0.26, 0.26]$ . Moving $0.3$ m sideways shifts the near plane $48$ pixels at $f = 800$ but the far only $24$ , which is the parallax MPI captures.

Watch Out For

Wide baselines exposing empty layers

A $2$ m sideways move shows gaps where no plane held paint. The symptom is stretched rubber edges. Keep MPI to small moves under $0.5$ m or switch to NeRF.

The Quick Version

MPIs stack fixed-depth RGBA planes for fast warping-based rendering.
Alpha blending back to front handles soft occlusion correctly.
Use MPI for nearby views on phones and photo effects.
Do not use it for wide orbits; discrete planes tear.
Thirty-two planes from $2$ m to $60$ m balance quality and memory.