CosFace Margin Face Loss
CosFace subtracts a fixed margin from the true identity cosine score, forcing every face to win its class by a clear gap instead of a whisker.
Why Does This Exist?
Softmax accepts whisker wins: a face beating its rival class by 0.001 counts as correct and teaches nothing about tighter clusters. Open-set verification then inherits sprawling classes with fuzzy borders. CosFace (Large Margin Cosine Loss) exists to demand daylight: subtract a fixed margin m from the target cosine before scaling, so training only rests when every face clears its rivals by the full gap. It is the cosine-space sibling of ArcFace; embeddings themselves are covered in FaceNet.
Think of It Like This
A high-jump bar, not a photo finish
Softmax is a race where leaning across first by a nose wins the gold. CosFace is a high jump where the bar sits m higher for the favourite: clearing it by a hair still fails unless the jump beats the raised bar. Faces must clear their own raised bar while rivals jump at standard height. The analogy stops at the geometry: bars are heights, while CosFace margins live in cosine space on a normalized hypersphere.
How It Actually Works
Embeddings and class weights are L2-normalized so logits are cosines scaled by s. The target logit becomes s x (cos(theta) - m) while rival logits stay s x cos. The fixed subtraction hits hardest where it matters: mid-range cosines lose a bigger fraction of their score than near-perfect ones, pushing the whole class inward. Typical settings use m around 0.35 with s near 64.
A worked gap
Scale s = 64, margin m = 0.35. A face with cos(theta) = 0.8 toward its class scores 64 x (0.8 - 0.35) = 28.8, while a rival at cosine 0.5 scores 32.0 and currently wins. The network must drag the face to cos(theta) above 0.85 before the target retakes the lead (64 x 0.5 = 32.0), which is exactly the enforced daylight. Easy faces already near 0.95 feel little pressure; borderline faces do the learning.
Watch Out For
Copying margin values across normalizations
CosFace margins assume normalized cosine logits; applying the same m to unnormalized dot products either does nothing or explodes training. The symptom is a margin that seems to have no effect at any setting. Fix it by verifying both embeddings and weights are unit length before the loss runs.
The Quick Version
- CosFace subtracts margin m from the target cosine inside softmax.
- Every face must beat its nearest rival by the full gap, tightening clusters.
- Typical values sit near m = 0.35 with scale s = 64.
- It complements ArcFace: cosine versus angular margin.
- Normalization of embeddings and weights is mandatory for the margin to mean anything.