Human Mesh Recovery from Photos

Human mesh recovery regresses SMPL pose and shape directly from pixels so one photo yields a posed 3D body.

Why Does This Exist?

Fitting SMPL by optimization per photo takes minutes and needs good starts. HMR (2018) trains an encoder to regress $(\theta, \beta)$ plus camera in one forward pass from a $224 \times 224$ crop. A discriminator keeps poses plausible when joints are hidden. This page covers direct regression; per-person polish still uses fitting on top.

Think of It Like This

A tailor who guesses sizes at a glance

An expert tailor glances at a customer and calls out neck, chest, and inseam without tape. Years of customers form the prior that makes one glance enough. Tape still checks the final suit.

The encoder is the glance and the discriminator is experience. The analogy stops at guarantees: guesses need joint checks, not blind trust.

How It Actually Works

ResNet features $f \in \mathbb{R}^{2048}$ feed iterative regressors outputting $\theta \in \mathbb{R}^{72}$ , $\beta \in \mathbb{R}^{10}$ , camera $(s, t_x, t_y)$ . Loss sums 3D joint error where mocap exists, 2D reprojection $\lVert \Pi(J) - j_{gt} \rVert^2$ everywhere, plus adversarial $\log D(\theta, \beta)$ . Weak perspective $\Pi$ suffices for crops.

Worked example

A crop shows a sideways lunge. Initial mean gives knee error $22$ pixels. After three regressor iterations, hip lands within $3$ pixels, knee within $5$ , ankle within $6$ . Shape $\beta_2 = -0.8$ slims thighs $2$ cm. Discriminator score $0.81$ marks the pose plausible. Full inference runs in $30$ ms against $2$ minutes for pure fitting.

Watch Out For

Truncated crops biasing shape tall

A crop cutting the head makes the weak-perspective scale overestimate height by $8$ cm. The symptom is giants in crowd edges. Pad crops to full torso or drop truncated people from shape metrics.

The Quick Version

HMR regresses SMPL numbers directly from a person crop.
Joint reprojection supervises everywhere; mocap supervises where present.
Adversarial priors keep hidden limbs plausible.
Do not trust shape from cut-off crops; joints stay better than girth.
One forward pass replaces minutes of per-photo fitting.