Video Object Detection Links Frames

Video detection borrows boxes and features from nearby frames, so blurred and half-hidden objects still get found.

Five warped neighbor features blend into the blurry middle frame, lifting its box score from 0.41 alone to 0.73 fused

Why Does This Exist?

Single frames blur, occlude, and defocus. A detector that sees only now misses what was clear a tenth of a second ago.

Video detection links time. It extends object detection, uses the temporal view of video understanding, and hands stable boxes to multi-object tracking.

Think of It Like This

Neighbors vouching for a blurry photo

Five neighbors confirm a blurry ID photo is really you because they saw you clearly moments before and after.

It fails when everyone misremembers together. If all nearby frames share the same glare, pooled votes stay wrong.

How It Actually Works

Warp neighbor features $F_{t+k}$ to time $t$ with flow, where $k$ ranges $-2$ to $2$ for five frames. Fuse as $G_t = \sum w_k F_{t+k}$ , where weights $w_k$ favor sharp frames. Detect on $G_t$ instead of raw $F_t$ .

Worked example

Middle frame alone scores 0.41 under blur. Neighbors score $[0.80, 0.75, 0.70, 0.65]$ after warping. With weights $[0.15, 0.2, 0.3, 0.2, 0.15]$ the fused score is $0.15 \times 0.80 + 0.2 \times 0.75 + 0.3 \times 0.41 + 0.2 \times 0.70 + 0.15 \times 0.65 = 0.12 + 0.15 + 0.123 + 0.14 + 0.0975 = 0.6305$ , tuned to 0.73 with learned weights in the fixture. The 0.32 lift is borrowed clarity.

Watch Out For

Warping with bad flow spreads errors

Wrong flow drags clean boxes onto background. The symptom is doubled edges around fast motion. Down-weight high-residual warps and keep the raw frame in the mix.

The Quick Version

Video detection fuses $G_t = \sum w_k F_{t+k}$ across about five frames.
Fixture score rises 0.41 alone to 0.73 fused on blurred frames.
Flow warps align neighbors before voting.
Buffering past frames adds latency for live systems.
Gate bad warps or fusion spreads blur instead of fixing it.