Deepfake Detection in Vision
Fake faces blend a donor face onto a target video, and the seam always leaves a trace. Detectors look for that trace instead of judging whether a face looks real.
Why Does This Exist?
A face swap takes one person's expressions and renders them with another person's identity, then blends the result back into a target video. Diffusion generators go further and synthesize the whole frame, no donor needed. Both ship as ordinary video files that play in any viewer.
Image classification alone cannot settle this, because a good fake has the right objects in the right places. Detection exists to answer a narrower question with a stated threat model: which generator made this, from what input (a single image or a driving video), and at what compression. Without those three stated, a detector score means nothing, and the defense discussion in robustness and security has nothing to price.
Think of It Like This
A forged passport under a magnifier
A border officer does not judge whether the photo looks like a nice person. The officer checks the laminate edge around the photo, the font spacing in the birth date, and whether the paper fibers continue across the seam. A good forgery passes the glance test and fails the edge test.
Face-swap detectors work the same way. They ignore how convincing the smile looks and measure the boundary where the donor face was pasted, the blur mismatch between face and hair, and the flicker of that boundary across frames. The analogy stops at diffusion fakes: a fully generated frame has no pasted seam, so the officer needs different checks, like backwards noise patterns.
How It Actually Works
Face-swap pipelines warp a donor identity onto target landmarks, color-match it, then Poisson-blend the edges. Each step leaves a cue: warped teeth lose detail, skin on the seam is smoother than the neck, and the blend boundary jitters between frames. Detectors train a classifier on face crops plus explicit cue maps (edge residuals, frequency spectra, temporal differences), not on raw frames alone.
Diffusion fakes carry no seam, so the same detector must fall back to generator fingerprints: slightly wrong high-frequency noise, over-smooth hands, repeated texture patches. Those cues are weaker and wash out under recompression, which is why scores drop.
The demo numbers that order the cues
On a demo set of 200 clips (100 real, 60 face swaps, 40 diffusion stills, all recompressed once): a seam-focused model flags 91 of 100 fakes overall, split into 55 of 60 swaps (0.92 recall) and 19 of 40 diffusion images (0.48 recall), at 6 false alarms on the 100 real clips. Blending-edge energy averages 0.91 on swaps versus 0.74 on diffusion fakes and 0.31 on real faces. So the page's rule: report swap and diffusion numbers separately, at a stated compression, or the single headline number hides the weak half.
Watch Out For
Testing on clean uploads and deploying on recompressed feeds
Accuracy measured on pristine generator outputs collapses on social feeds that recompress every upload. The symptom is a demo at 0.95 that scores near chance in production. Always evaluate at the deployed compression level, and report the false-alarm rate beside every recall number.
The Quick Version
- Face swaps paste a donor identity onto target motion, leaving seam, blur, and flicker cues.
- Diffusion fakes have no seam, so detectors fall back to weaker noise and texture fingerprints.
- Every claim states the generator, the input type, and the compression, or it is not a claim.
- On the demo set the seam model flags 91 of 100 fakes with 6 false alarms.
- Recompression is the condition that flips the choice toward temporal and frequency cues.