Face Reconstruction from One Photo

Face reconstruction splits one portrait into identity shape, expression, pose, and light for editable 3D faces.

Why Does This Exist?

Portraits mix who you are with how you smile and where light falls. Avatars and recognition need identity alone. Face reconstruction fits a 3D morphable model to landmarks plus pixels, separating identity $\alpha$ , expression $\delta$ , pose $R, t$ , and light $L$ . Face detection and recognition finds the face; this page builds its 3D shape.

Think of It Like This

Separating actor, grin, stage angle, and spotlight

One theater photo shows an actor grinning under a side spotlight from row five. The actor stays the same across shows, while grins, seats, and lights change nightly. Billing needs the actor name, not the grin.

Identity is the actor and the rest are nightly variables. The analogy stops at detail: pores and wrinkles exceed the morphable basis.

How It Actually Works

Sixty-eight landmarks anchor eyes, nose, and mouth within $3$ pixels. Photometric loss matches rendered pixels under Spherical Harmonics light with $27$ coefficients. Identity $\alpha \in \mathbb{R}^{80}$ stays fixed across photos of one person while $\delta$ , $R$ , $L$ vary per photo.

Worked example

A smiling portrait gives mouth corner error $8$ pixels under neutral identity. Adding expression $\delta$ with smile weight $0.7$ drops mouth error to $2$ pixels while nose stays at $1$ . Albedo recovers skin $[0.72, 0.58, 0.48]$ after dividing side light $0.8$ . Switching $L$ to frontal relights without changing $\alpha$ , so identity distance stays $0.15$ versus $0.9$ across people.

Watch Out For

Expression leaking into identity

Fitting one smiling photo with free identity bakes the grin into $\alpha$ , so neutral renders still smile. The symptom is same-grin avatars. Fit three expressions jointly or pin $\alpha$ from a neutral photo.

The Quick Version

One portrait factors into identity, expression, pose, and light.
Sixty-eight landmarks anchor alignment within pixels.
Identity stays fixed while expression and light vary per photo.
Do not bake one grin into identity; fit multiple expressions.
Relighting edits light while recognition reads identity alone.