DSO Direct Sparse Odometry Guide

Track a few thousand bright pixels with full calibration. DSO shows direct sparse odometry can beat features when photometry is modeled right.

A sliding window of keyframes jointly refines sparse depths and brightness-aware poses

Why Does This Exist?

LSD-SLAM proved direct tracking works but kept mapping semi-dense and loosely coupled. DSO exists to make direct estimation sparse, calibrated, and jointly optimized: a fixed window of keyframes refines poses, intrinsics, brightness parameters, and about two thousand active points together with photometric bundle adjustment. It beats feature systems on well-calibrated sequences with repeating texture that fools descriptors.

This page covers windowed direct odometry without loop closure. Feature loops are ORB-SLAM; loop repair is loop closure. Basics of frame motion are visual odometry.

Think of It Like This

Proofreading one paragraph at a time

An editor proofreads a sliding window of seven paragraphs, fixing spelling and layout together, then drops the oldest paragraph once the next one arrives. The book stays consistent without re-reading all pages each time.

Keyframes are paragraphs and photometric bundle adjustment is the joint fix. The analogy stops at revisits: the editor never reopens chapter one, so old drift never gets repaired.

How It Actually Works

Point selection keeps about $2000$ active pixels with high gradient spread evenly across the image. Each residual compares brightness across frames with affine transfer $r = (I_j - b_j) - \frac{t_j e^{a_j}}{t_i e^{a_i}} (I_i - b_i)$ , where $a$ and $b$ model log exposure and bias per frame. A sliding window of about seven keyframes optimizes poses, inverse depths, intrinsics, and brightness terms together with Gauss-Newton, marginalizing the oldest keyframe with the Schur complement to keep priors without growing cost.

Worked numbers

Seven keyframes hold $2000$ points with $14000$ residuals. Mean photo error falls from $9.5$ to $0.3$ gray levels after convergence. Inverse depth $0.5$ (depth $2.0$ m) with variance $0.02$ tightens to variance $0.004$ after six observations, while a weak point keeps variance above $0.05$ and gets marginalized out.

Code

1def photometric_cost(i_j: float, b_j: float, i_i: float, b_i: float, gain: float) -> float:2    return (i_j - b_j) - gain * (i_i - b_i)  # affine brightness compensated error3
4
5print(round(photometric_cost(120.0, 2.0, 118.0, 1.0, 1.0), 3))  # -> 1.0

Watch Out For

Running DSO uncalibrated

DSO trusts the photometric model, so wrong vignetting or response curves bias every residual. The symptom is confident tracking with bent geometry on cheap cameras. Calibrate intrinsics, vignetting, and response first, or use a feature method that tolerates calibration slop.

The Quick Version

DSO is direct sparse odometry over raw brightness, not descriptors.
About $2000$ spread points in seven keyframes carry the estimate.
Affine brightness terms absorb exposure change per frame.
Windowed photometric bundle adjustment refines poses and depths together.
Marginalization keeps priors from dropped frames without regrowing cost.
There is no loop closure, so long loops still drift.