Classical Object Tracking

Classical trackers predict where a box moves next from its past motion, then correct with the new frame. That predict-then-correct loop holds one identity per object without running a detector on every frame.

A box at 90 moving 12 pixels per frame is predicted at 102, then a detection at 104 corrects it to 103

Why Does This Exist?

Detecting an object in every frame works, but it is expensive and it answers the wrong question: a detector says a car is here, not that it is the same car as last frame's. Tracking connects sightings into trajectories with stable identities, at a fraction of the per-frame cost. Detections usually come from a per-frame step such as background subtraction on a fixed camera. This page maps how the motion and appearance halves fit; the single-object loop here grows identities and data association in multi-object tracking.

The one idea you need first

Two frames of the same scene differ mostly by motion: each pixel in frame $t-1$ appears shifted somewhere nearby in frame $t$ . Optical flow measures those shifts, sparsely at tracked points with Lucas-Kanade or densely across the frame. Trackers either model the box's motion directly or ride these pixel shifts; both beat re-searching the whole image.

Think of It Like This

Tracking a player after a blink

You are watching a football match and tracking one player. When you blink, you do not rescan the whole field on opening your eyes. You know their speed and direction, so you look slightly ahead of where they were and confirm the shirt number. The glance ahead is the prediction step; checking the shirt is the correction step.

The analogy stops at crowded scenes: you recognise a face, while a colour tracker happily jumps to a similarly dressed stranger who crosses the predicted spot.

How It Actually Works

Classical tracking splits into motion prediction and appearance matching, fused once per frame.

Motion prediction

The Kalman filter keeps position plus velocity, predicts the box forward each frame, then blends the prediction with the new detection. Through occlusions it coasts on prediction alone. When one bell curve cannot hold the belief, particle filters keep a cloud of guesses and resample around the winners.

Appearance matching

Mean shift climbs the target's colour-similarity hill to re-centre the box with no motion model at all, and CamShift adds resizing from the colour mass it finds. Template matching slides the original patch over the predicted neighbourhood and takes the best score. Flow supplies the pixels underneath both.

Worked example

A box edge sits at $x = 90$ pixels moving at $v = 12$ pixels per frame with $dt = 1$ . The prediction is $x + v \cdot dt = 90 + 12 = 102$ . The new frame detects the edge at $104$ . Blending halfway gives $102 + 0.5 \times (104 - 102) = 103$ , the updated position. Prediction kept the search window tiny; the detection kept the drift honest.

Code

The predict-then-correct loop in its simplest form:

1x, v, dt = 90.0, 12.0, 1.02pred = x + v * dt3print(pred)  # -> 102.04
5updated = pred + 0.5 * (104.0 - pred)  # halfway blend with the detection6print(updated)  # -> 103.0

Watch Out For

A colour model that jumps objects

Trackers built purely on colour histograms drift when the target enters shadow or a similarly coloured object crosses behind it. The symptom is a box that slides off the target onto a red truck and never comes back. Gate matches by predicted position as well as appearance, and kill the track when both disagree for several frames.

A template that never refreshes

A patch cut from frame one goes stale as the object rotates, scales, or relights, so match scores decay until the tracker latches onto background clutter that happens to resemble day one. The symptom is slow confident drift over tens of frames. Refresh the template from confident matches, and cap how far any single update may move it.

The Quick Version

Tracking predicts each box forward from past motion, then corrects with the new frame's evidence.
A box at 90 moving 12 per frame predicts 102, and a detection at 104 corrects it to 103.
Kalman filters carry position plus velocity and coast through occlusions on prediction alone.
Mean shift and template matching re-find appearance without any motion model.
Colour-only models jump to lookalikes, and frozen templates decay into background drift.