TSN Votes Across Video Segments

TSN splits a video into a few distant segments, scores one frame from each, then averages the votes into one video decision.

TSN cuts a long video into 3 segments, scores one frame from each at 0.7, 0.5, and 0.9, and averages them to 0.70

Why Does This Exist?

Dense 3D clips watch seconds, not minutes. A two-minute action spreads its evidence across the whole timeline, and a 16-frame window sees only one slice of it.

TSN samples sparsely across the full video instead. It supports video classification, feeds action recognition, and stays cheaper than dense volumes like two-stream networks by scoring few frames well spaced.

Think of It Like This

Three jurors watching different acts

Three jurors each watch one act of a play and vote. No juror sees everything, but together they cover the whole show.

It fails when the key moment falls between sampled acts. Sparse votes miss brief events that dense watching would catch.

How It Actually Works

Split video $V$ into $K = 3$ equal segments, where $K$ sets temporal coverage. From segment $k$ pick one frame $f_k$ and score it $s_k = F(f_k)$ , where $F$ is the shared image model. The video score is the mean $S = (s_1 + s_2 + s_3) / 3$ , where averaging makes every segment count equally.

Worked example

Scores come back $s_1 = 0.7$ , $s_2 = 0.5$ , $s_3 = 0.9$ . Then $S = (0.7 + 0.5 + 0.9) / 3 = 2.1 / 3 = 0.70$ . One strong 0.9 segment plus two middling votes still passes threshold 0.5. A dense 16-frame clip stuck in segment two would see only 0.5 and call it negative.

Watch Out For

Averaging away a brief action

A two-second dive inside a two-minute swim averages down to noise. The symptom is high per-segment peaks with a flat video mean. Replace plain mean with top- $k$ pooling or learned weights when actions are brief.

The Quick Version

TSN splits video into $K = 3$ segments and scores one frame each.
Video score $S$ is the mean of segment scores, here 0.70 from 0.7, 0.5, 0.9.
Sparse sampling covers minutes while dense clips cover seconds.
Shared weights keep it cheap, and flow streams add motion.
Brief actions need top- $k$ pooling, not plain averaging.