Cityscapes Street Segmentation

Street scenes with dense pixel labels for road, car and person, the ground that made urban segmentation comparable.

Cityscapes maps each street photo to a dense pixel mask, so road, car and person regions train urban segmentation

Why Does This Exist?

Driving needs pixel answers: where is road, where is sidewalk, which pixels belong to a person. Image-level labels cannot supply that. Cityscapes collected street photos across towns and labeled them densely per pixel, so semantic segmentation models could train and compare on the same urban ground.

This page covers purpose, dense I/O roles and appropriate use. Architectures live in U-Net and DeepLab.

Think of It Like This

Coloring every part of a street photo

Print a street photo and hand out crayons with a fixed key: gray for road, blue for car, red for person. Every pixel gets a color, no blanks.

That colored copy is the label. The model must reproduce the coloring from the plain photo alone.

How It Actually Works

Purpose is urban semantic and instance segmentation. Input is one street photo, output is one label per pixel plus instance tags where used. Appropriate use is benchmarking urban segmenters and pretraining driving perception. Say less about exact split counts here and verify the official release before quoting them.

Toy pixel walkthrough

Take a tiny 10-pixel strip where 6 pixels are road and 4 are car. A model labels 5 road right, 1 road as car, and 3 car right with 1 car as road. Road IoU is $5 / (5 + 1 + 1) = 5 / 7 \approx 0.714$ . Mean over classes averages such ratios, which punishes both misses and false paint equally.

Code

Pixel IoU on toy masks, the unit behind mean IoU.

1import numpy as np2
3
4def iou(a: np.ndarray, b: np.ndarray) -> float:5    inter = np.sum((a == 1) & (b == 1))  # both call it road6    union = np.sum((a == 1) | (b == 1))  # either calls it road7    return float(inter / max(union, 1))8
9
10true = np.array([1, 1, 1, 1, 1, 1, 0, 0, 0, 0])11pred = np.array([1, 1, 1, 1, 1, 0, 0, 0, 0, 1])12print(f"{iou(true, pred):.3f}")13# -> 0.714

Five shared road pixels over seven claimed gives 0.714.

Watch Out For

Training on one set of towns and shipping everywhere

Street layout, markings and light shift by city, so a model tuned to training towns drops elsewhere. The symptom is clean validation with weak new-city video. Fix it by holding out whole towns and testing night and rain slices separately.

The Quick Version

Cityscapes maps one street photo to one dense pixel mask for urban classes.
Input is a photo, output is a per-pixel label map with instance tags where used.
Toy 5 shared over 7 claimed gives road IoU about 0.714.
Use it to benchmark urban segmentation and driving perception starts.
Verify official splits before quoting counts, and test held-out towns.