DAVIS Video Segmentation Set

Short videos with accurate object masks per frame, the crisp ground for tracking masks through motion and occlusion.

DAVIS pairs each video frame with an object mask, so the same region tracks through motion and occlusion

Why Does This Exist?

Box tracking leaves boundaries vague and occlusion handling untested. Video object segmentation needs pixel masks that persist across frames. DAVIS supplied short videos with accurate per-frame masks, so video understanding could score boundary quality plus temporal stability.

This page covers mask persistence contract and appropriate use. Flow help is in optical flow.

Think of It Like This

Tracing the same runner each lap photo

Photograph a runner each lap and trace their outline every time, even when another runner briefly blocks them.

That traced series is DAVIS. Same person, every frame, exact edges.

How It Actually Works

Purpose is video object segmentation. Input is a short clip, outputs are per-frame masks with consistent IDs. Appropriate use is benchmarking mask propagation and interactive segmentation. Short length and few objects limit generality, so confirm on longer sets for deployment claims.

Toy mask walkthrough

Take a frame with 100 truth mask pixels where prediction shares 80, misses 20 and adds 10 false. IoU is $80 / (80 + 20 + 10) = 80 / 110 \approx 0.727$ . Averaging such frames prices both drift and boundary error.

Code

Frame IoU that averages across a clip.

1import numpy as np2
3
4def mask_iou(a: np.ndarray, b: np.ndarray) -> float:5    inter = np.sum((a == 1) & (b == 1))6    union = np.sum((a == 1) | (b == 1))7    return round(float(inter / max(union, 1)), 3)8
9
10print(mask_iou(np.ones(80), np.ones(80)))11# -> 1.0

Identical 80-pixel masks give 1.0, while the toy 80 over 110 gives about 0.727.

Watch Out For

Memorizing short-clip appearance

Few frames per video let models overfit colors. The symptom is clean DAVIS with weak long-video drift. Fix it by testing long occlusions and unseen objects separately.

The Quick Version

DAVIS maps short clips to per-frame masks with consistent IDs.
Input is a clip, outputs are masks that persist through occlusion.
Toy 80 shared over 110 claimed gives about 0.727 IoU.
Use it for crisp VOS baselines and propagation tests.
Confirm on longer videos before broad claims.