Anomaly Detection in Video

Video anomaly detection learns routine motion and flags clips that break the routine, like a fall or a wrong way runner, without naming every action.

Normal clips rebuild with error near 0.08 while a fall spikes to 0.47, so a 0.20 cut separates routine from alert

Why Does This Exist?

Anomalies are rare and open ended: no dataset lists every future fall, fight, or spill. Supervised classifiers wait for labels that never arrive in time. Corridors stream hours of routine walking that define normal better than any list.

Video anomaly detection exists for that one class video setting. Input is clips from fixed scenes. Output is a per frame or per clip strangeness score. The video base is video understanding; named motions are action recognition. Pixel motion is optical flow.

Use it when normal dominates and anomalies surprise. Use supervised action models when the target list is closed and labeled.

Think of It Like This

A hallway hum you stop hearing

You work beside a hallway and stop hearing routine steps. A thud, a sprint, or sudden quiet pops out without you naming footstep types. You check only the pop.

Normal training is learning the hum. Reconstruction error is the pop loudness. The analogy stops at adaptation: you relearn a new hum in days while a frozen model keeps flagging the new normal until retrained.

How It Actually Works

Train an autoencoder or future predictor on normal clips only. At test time score each frame by rebuild failure $e = |x - \hat{x}|$ averaged over pixels, or by next frame prediction miss. Smooth scores over a second, then threshold. Clean hallway rebuilds at $e \approx 0.08$ . A fall with fast limbs and occlusions rebuilds at $0.47$ . A cut at $0.20$ passes routine and pages the fall with its time stamp.

Worked example

Ten normal clips score $0.06$ to $0.11$ , mean $0.08$ . A wrong way runner scores $0.38$ . A fall scores $0.47$ . With cut $0.20$ , both anomalies fire and zero normal clips fire. Precision $= 2/2 = 1.0$ on this night. Lowering the cut to $0.07$ would fire $4$ normal jitters alongside, dropping precision to $2/6 \approx 0.33$ for no new finds.

Watch Out For

Rare normal flagged as anomaly

A cleaner with a cart at 4am never appeared in daytime training. Error spikes to $0.35$ and pages a ghost. The symptom is dawn false alarms. Fix it with round the clock normal coverage and scene specific thresholds.

Scene change mistaken for behavior change

A moved camera or new poster shifts every pixel and lifts all errors by $0.10$ . The symptom is a solid block of alerts after maintenance. Fix it by re baselining on the new view and by gating on motion magnitude, not raw pixels alone.

The Quick Version

Video anomaly learns normal motion and scores deviation per clip.
Score is rebuild or prediction error $e = |x - \hat{x}|$ .
The fixture flags $0.47$ and $0.38$ against normal $0.08$ at a $0.20$ cut.
Smooth over a second and stamp the alert time for review.
Cover all hours and re baseline after camera moves or nights mislead.