Scene Reconstruction from Images

Scene reconstruction fuses many photos into one consistent 3D map of walls, furniture, and free space.

Why Does This Exist?

One depth map covers one viewpoint. Robots and AR need whole rooms. Structure from motion recovers poses; dense fusion integrates each depth frame into a TSDF volume; marching cubes extracts the mesh. This page covers the static-room pipeline before dynamics and semantics.

Think of It Like This

Building a clay room from traced windows

You walk around a dark room shining a flashlight through traced windows, pressing clay where each beam lands. Returning to the start, you align the last press with the first to close drift. The clay room outlives any single view.

Frames are beams and TSDF is clay. The analogy stops at loop closure: clay shows seams while optimization spreads the correction.

How It Actually Works

Poses $T_i$ come from visual tracking with reprojection under $1$ pixel. Each depth frame integrates into TSDF with truncation $\tau = 0.1$ m and weight capped at $100$ hits. Loop closure detects revisits by visual bags of words and runs pose-graph optimization cutting drift from $0.5$ m to $0.03$ m over $100$ m paths.

Worked example

A $5 \times 4$ m room scanned in $200$ frames fills $400{,}000$ TSDF voxels at $0.02$ m. A wall point seen in $40$ frames averages distance $0.01$ m with weight $40$ . Without loop closure the far wall duplicates $0.3$ m apart; after closure the duplicates merge within $0.02$ m and marching cubes yields one wall.

Watch Out For

Moving people baked into walls

A person standing $30$ seconds writes density $3.0$ into the wall volume. The symptom is ghost limbs in offices. Mask people with segmentation before fusion or keep a transient branch.

The Quick Version

Poses plus depth fusion build whole rooms from photo sets.
TSDF truncation near $0.1$ m bounds updates around surfaces.
Loop closure cuts long-path drift from decimeters to centimeters.
Do not fuse people into static maps; mask dynamics first.
Marching cubes extracts the single mesh for navigation and AR.