Face Landmark Alignment

Face landmarks pin sixty-eight points along eyes, brows, nose, and jaw so AR filters, gaze tracking, and face recognition align precisely.

Face landmark detection traces eyes brows nose and jaw with pinned points so downstream tasks start aligned.

Why Does This Exist?

A face box tells filters and recognizers where the face is but not where the eyes point or the mouth opens. Sixty-eight-point landmarks (Multi-PIE / 300-W convention: jaw $17$ , brows $10$ , nose $9$ , eyes $12$ , mouth $20$ ) convert the box into aligned geometry. Eye-aspect ratio from six eye points drives blink detection; five-point sparse sets align recognition embeddings; dense $468$ -point meshes (MediaPipe Face Mesh) anchor AR makeup.

Think of It Like This

Tent pegs before raising canvas

Pitching a tent starts with pegging corners and guylines, not draping canvas over grass. Landmarks are the pegs: eyes and jaw corners stake the face shape, then recognition and AR canvas stretches over them. A shifted peg wrinkles the whole tent.

Where it stops: pegs mark the surface, not the skull, so extreme expressions move pegs while identity stays put.

How It Actually Works

Classical cascaded regressors (dlib Kazemi-Sullivan, 2014) refine points from the box mean shape. Modern heads regress $68 \times 2$ coordinates directly or predict heatmaps per point, trained with inter-ocular-normalized error (NME): mean point distance divided by eye-corner distance. Five percent NME is a solid operating point. Alignment warps faces to canonical eye positions before recognition.

Worked example

Eye corners at $(100, 100)$ and $(150, 102)$ : inter-ocular distance $\approx 50$ . Predicted nose tip off by $(3, 4)$ : error $5$ . NME $= 5 / 50 = 10$ percent, poor. Refined prediction off by $(1, 1)$ : error $\approx 1.41$ , NME $\approx 2.8$ percent, good. Eye-aspect ratio from vertical lid distances $6$ and $6$ over horizontal $20$ : $(6 + 6) / (2 \times 20) = 0.3$ open; below $0.2$ sustained reads as a blink.

Code

1import math2
3# NME and eye-aspect ratio.4nme = math.dist((103, 104), (100, 100)) / 505ear = (6 + 6) / (2 * 20)6print(round(nme * 100, 1), round(ear, 2))7# -> (10.0, 0.3)

Watch Out For

Profile poses breaking 68-point models

Frontal-trained models hallucinate occluded cheek points past $45$ degrees yaw. Symptom: jawline sliding off the face. Fix: use pose-specific models or 3D-aware mesh heads for profile ranges.

Expression shift poisoning recognition

Smiling mouth points move while identity does not. Symptom: expression-sensitive mismatches. Fix: align on stable eye-nose anchors and prefer expression-robust embeddings downstream.

The Quick Version

Sixty-eight points trace jaw, brows, nose, eyes, and mouth by convention.
NME normalized by eye distance is the comparability metric.
Five sparse points align recognition; dense meshes anchor AR.
Eye-aspect ratio turns six lid points into blink detection.
Profile yaw and heavy expression need specialized handling.