3D Morphable Models for Faces

3D morphable models compress thousands of face scans into identity and expression dials that stay plausible by construction.

Why Does This Exist?

Free face meshes fold into monsters. The Basel model (2009) PCA-compresses $200$ scans into mean plus bases, so any blend stays face-like. FLAME adds articulated jaw and eyelids. Face reconstruction optimizes these dials instead of vertices.

Think of It Like This

A face mixing board with labeled sliders

A sound board blends instrument tracks into any song without playing each note. A face board blends $80$ identity tracks plus $64$ expression tracks into any face without sculpting pores. Pushing all sliders wild still sounds like music, not noise.

Bases are tracks and coefficients are faders. The analogy stops at coverage: boards miss tattoos and scars outside the scans.

How It Actually Works

Shape is $S = \bar{S} + A_{id}\alpha + A_{exp}\delta$ with $A_{id} \in \mathbb{R}^{3N \times 80}$ and $A_{exp} \in \mathbb{R}^{3N \times 64}$ for $N \approx 53{,}000$ vertices. Texture is $T = \bar{T} + B\beta$ . Fitting minimizes landmark plus photometric error with Mahalanobis prior $\lVert \alpha \rVert^2$ keeping faces near mean.

Worked example

Mean nose length $5.2$ cm with first identity basis adding $0.4$ cm per unit. Setting $\alpha_1 = 1.5$ gives $5.2 + 0.6 = 5.8$ cm. Expression smile basis moves mouth corners $+0.8$ cm at $\delta = 1.0$ . Landmark error drops from $9$ to $2$ pixels. Prior cost $1.5^2 = 2.25$ keeps the face within scanned variation.

Watch Out For

Out-of-distribution faces forced onto the board

Scars and rare craniofacial shapes project to the nearest plausible face, erasing the very detail doctors need. The symptom is clean but wrong clinic meshes. Flag high prior cost above $8$ for human review.

The Quick Version

Morphable models blend scan bases for identity plus expression.
Eighty identity and $64$ expression dials span most faces.
Priors keep fits human even from one photo.
Do not use them for tattoos, scars, or rare anatomy without extensions.
FLAME adds jaw and eye articulation over the Basel base.