LSD-SLAM Direct Semi-Dense Maps

Skip corners and use every pixel with a gradient. LSD-SLAM builds semi-dense depth from brightness itself, which survives where corners vanish.

High-gradient pixels carry depth while flat pixels stay empty in the semi-dense map

Why Does This Exist?

Corner detectors discard most of the image, which hurts in offices with long clean edges but few corners. LSD-SLAM exists to use all informative pixels directly: it tracks and maps from brightness over high-gradient regions, producing semi-dense depth where feature maps stay sparse. It proved direct monocular SLAM could run large scale on a CPU.

This page covers the direct semi-dense design. Feature sparsity is ORB-SLAM; sparse direct odometry is DSO. Tracking drift repair is loop closure.

Think of It Like This

Reading edges instead of street signs

A driver in fog ignores distant signs and steers by the road edges sliding past the windshield. Signs are rare but edges are everywhere, so the edge reader keeps going where the sign reader stalls.

Gradients are road edges and corners are signs. The analogy stops at exposure: road paint never changes brightness while camera auto-gain shifts every frame and must be modeled.

How It Actually Works

Tracking aligns the new frame directly against the keyframe by minimizing photometric error $r = I_2(\pi(T X)) - I_1(u)$ over pose $T$ , using all pixels with gradient magnitude above threshold. Mapping propagates each high-gradient pixel into a depth hypothesis with variance, then refines it with small-baseline stereo across frames. Keyframes spawn on distance and appearance change; a similarity transform $\mathrm{Sim}(3)$ with seven degrees of freedom corrects scale drift at loop closures, since monocular direct maps drift in scale as well as pose.

Worked numbers

A $640 \times 480$ frame keeps about $60000$ high-gradient pixels out of $307200$ . Photometric residuals start near $12.0$ gray levels and settle near $0.7$ after alignment. Depth variance on a strong edge falls from $0.25$ m to $0.03$ m after eight observations with $0.1$ m baseline each.

Code

1def photometric_residual(i2: float, i1: float) -> float:2    return i2 - i1  # brightness difference after warping by pose and depth3
4
5print(photometric_residual(118.7, 118.0))  # -> 0.7000000000000028

Watch Out For

Ignoring exposure change in direct cost

Auto-exposure shifts brightness by several gray levels between frames, which the raw photo cost reads as motion. The symptom is steady drift in brightness-varying rooms. Estimate an affine brightness transfer $a I + b$ per frame or the tracker chases light, not geometry.

The Quick Version

LSD-SLAM uses high-gradient pixels directly instead of corners.
Tracking minimizes photometric error over pose from brightness.
Mapping grows semi-dense depth with per-pixel variance filtering.
Scale drifts freely in monocular mode and needs Sim(3) correction.
About one fifth of pixels carry depth; flat areas stay empty.
Exposure modeling is required, not optional, for direct methods.