Hand Mesh Recovery from Images

Hand recovery fits an articulated hand model to high-resolution crops because fingers need detail that body crops erase.

Why Does This Exist?

Body crops give hands about $30$ pixels, so fingers blur into mittens. Hands also articulate $20+$ degrees of freedom with self-occlusion. Dedicated hand recovery crops at $128$ pixels or more and fits MANO (shape $\beta$ , pose $\theta$ for $16$ joints). SMPL-X attaches the result to the body.

Think of It Like This

Reading rings from a jewel close-up, not a street photo

You cannot read ring engravings from across the street. You walk up, take a macro shot, and trace each finger separately. The street photo finds the person; the macro reads the hand.

Body crops are street photos and hand crops are macros. The analogy stops at interaction: hands touch objects while rings sit still.

How It Actually Works

A detector crops the hand at $128 \times 128$ or higher. The net regresses MANO $(\theta, \beta)$ plus weak camera. Twenty-one joints project to $2$ D with loss under $4$ pixels on visible tips. Contact loss penalizes mesh interpenetration past $2$ mm for grasping hands.

Worked example

An open palm crop shows all tips. Index tip projects within $2$ pixels, pinky within $3$ , wrist within $2$ . Thumb flexion $\theta = 0.5$ rad matches the photo angle. Closing into a fist hides four tips; visible wrist plus silhouette keep error at $6$ pixels while hidden joints rely on prior with $12$ -degree uncertainty. That gap is the honest limit.

Watch Out For

Motion blur merging adjacent fingers

A $1/30$ s exposure during a wave smears index into middle finger. The symptom is webbed meshes on video frames. Raise shutter speed or drop blurred frames from fitting.

The Quick Version

Hands need dedicated high-resolution crops, not body-crop leftovers.
MANO compresses articulation into pose plus shape like SMPL does.
Twenty-one joints anchor fingers within a few pixels when visible.
Do not trust hidden finger angles; priors guess with wide uncertainty.
Contact terms stop grasping fingers passing through each other.