Dynamic NeRF for Moving Scenes

Dynamic NeRF adds a time input and motion field so one model renders moving people and pets from any angle.

Why Does This Exist?

Static NeRF blurs a waving hand into fog because one 3D point holds many colors over time. Video needs motion. Dynamic NeRF variants add time $t$ and either condition the field $F(p, t)$ directly or warp $p$ through a deformation $D(p, t)$ into a canonical frame. This page covers the warp design, which separates motion from appearance cleanly.

Think of It Like This

A flipbook traced onto one master drawing

Each flipbook page shows a runner in a new pose. You trace every page back onto one standing figure by noting how each limb moved, then color the master once. New poses render by bending the master forward again.

Canonical space is the master and deformation is the tracing. The analogy stops at topology: tracing cannot create or remove limbs, and neither can warps.

How It Actually Works

Deformation $D$ maps $(p, t)$ to canonical $p_c = p + \Delta(p, t)$ with $\Delta$ from a small MLP. The static field $F(p_c)$ gives $(\sigma, c)$ . Rendering marches in observed space but queries $F$ at $p_c$ . A rigidity prior keeps $\lVert \Delta \rVert$ small where optical flow is static.

Worked example

An arm point at $p = (0.30, 1.20, 0.10)$ m in frame $t = 0.4$ s warps by $\Delta = (-0.05, 0.10, 0.0)$ to $p_c = (0.25, 1.30, 0.10)$ m. Canonical density there is $4.0$ with color $[0.75, 0.60, 0.50]$ . Rendering that ray gives skin tone instead of fog. Background points get $\Delta \approx (0.001, 0.002, 0)$ and stay locked.

Watch Out For

Topology change breaking the warp

Opening a mouth or picking up a cup creates surfaces with no canonical partner, so the warp stretches rubber across the gap. The symptom is webbing between hand and cup. Use a per-time transient head for appearing objects.

The Quick Version

Time conditioning plus a warp field turns static NeRF into video NeRF.
Canonical space holds appearance; deformation holds motion.
Rigidity priors keep static background from swimming.
Do not use warps for appearing or vanishing objects; add transient branches.
A $10$ -second clip trains one model for any viewpoint at any frame.