ScanNet Indoor 3D Scans

Room scans with camera poses, meshes and semantic labels, the ground for indoor 3D segmentation and reconstruction.

ScanNet joins RGBD frames, poses and meshes with semantic labels, so indoor 3D trains on aligned scans

Why Does This Exist?

Photos alone underspecify rooms: walls hide backs, scales drift, occlusions confuse. Indoor 3D needs posed depth frames fused into meshes with labels. ScanNet scanned rooms with RGBD plus poses and released meshes with semantic tags, so point-cloud and 3D segmentation share one indoor ground.

This page covers scan roles and appropriate use. Geometry basics are in SfM.

Think of It Like This

A dollhouse built from phone scans

Walk through a room filming plus depth, then glue the views into a dollhouse with each chair and wall named.

That dollhouse is ScanNet. Views, shape and names stay aligned.

How It Actually Works

Purpose is indoor 3D segmentation and reconstruction. Inputs are RGBD frames with poses, outputs are meshes, voxels or point labels. Appropriate use is indoor 3D baselines and view-fusion tests. Homes and offices dominate, so warehouses and factories transfer weakly.

Toy voxel walkthrough

Take 20 voxels where 12 are chair and 8 are floor. A model labels 10 chair right, misses 2 chair and falsely paints 2 floor as chair. Chair IoU is $10 / (10 + 2 + 2) = 10 / 14 \approx 0.714$ .

Code

Voxel IoU for one class.

1import numpy as np2
3
4def voxel_iou(a: np.ndarray, b: np.ndarray) -> float:5    inter = np.sum((a == 1) & (b == 1))6    union = np.sum((a == 1) | (b == 1))7    return round(float(inter / max(union, 1)), 3)8
9
10print(voxel_iou(np.array([1]*10 + [0]*10), np.array([1]*10 + [0]*10)))11# -> 1.0

Matched voxels give 1.0, while toy 10 over 14 gives about 0.714.

Watch Out For

Assuming scan quality matches deployment

Handheld dense scans beat robot or glasses captures. The symptom is clean mesh scores with live holes. Fix it by testing sparse and noisy depth slices.

The Quick Version

ScanNet maps room scans to meshes plus semantic labels with poses.
Inputs are RGBD plus poses, outputs are 3D labels by voxel, point or mesh.
Toy 10 over 14 gives chair IoU about 0.714.
Use it for indoor 3D baselines and fusion tests.
Expect gaps on non-home spaces and sparse sensors.