3D Human Reconstruction from Images

3D human reconstruction lifts detected people into posed meshes by combining a body prior with image evidence.

Why Does This Exist?

Pose estimation gives skeletons, but clothing, contact, and shape need surfaces. Free-form meshes from one photo fold impossibly without priors. This umbrella page frames the family: SMPL and SMPL-X supply the body prior, human mesh recovery regresses it, hands and faces add detail, and 3D morphable models specialize faces.

Think of It Like This

Tailoring a suit onto a mannequin from photos

You pick a mannequin near the customer's size, pin it to match joint marks in front and side photos, then let out seams where silhouettes bulge. The mannequin stops elbows bending backward while photos set the true girth.

Body models are mannequins and joints are pins. The analogy stops at clothing: mannequins wear suits while bare priors need cloth layers.

How It Actually Works

Detection crops each person. A network regresses pose $\theta$ and shape $\beta$ for the prior plus camera $t$ . Joints $J(\theta, \beta)$ project to $2$ D and match detected keypoints with reprojection error under $5$ pixels. Silhouette IoU above $0.8$ refines girth. Hands and face use higher-resolution crops with the same loop.

Worked example

A $256 \times 256$ crop shows a runner. Initial mean pose gives mean joint error $18$ pixels. After regression, elbow lands within $3$ pixels, knee within $4$ , and silhouette IoU rises from $0.61$ to $0.84$ . Shape $\beta_1 = 1.2$ marks a broader torso than mean, shifting chest vertices $+4$ cm. That $4$ cm is measurable; finger bends under $5$ pixels are not.

Watch Out For

Depth flip on straight limbs

A straight arm pointing at the camera projects identically forward or backward. The symptom is elbows snapping mid-video. Add a pose prior against hyperextension and smooth temporally.

The Quick Version

Humans need prior-constrained meshes, not free vertices, from one photo.
Joints anchor pose within pixels; silhouettes set body girth.
Hands and faces need separate high-resolution crops to resolve.
Do not trust finger depth from wide shots; crop or use multiview.
Umbrella first, then SMPL, HMR, hand, face, and morphable details.