MiDaS Robust Monocular Depth Model

Train on many mixed datasets with one scale-free loss and depth works on unseen photos. MiDaS is the model that proved that mix-and-align recipe.

Mixed datasets align by scale and shift into one robust relative depth model

Why Does This Exist?

Depth datasets disagree on scale, units, and range, so training on one overfits to its camera. MiDaS exists to mix many datasets anyway: its scale-and-shift-invariant loss aligns each batch before scoring, letting indoor, outdoor, movie, and synthetic sources train one model that works on unseen photos. It became the default relative-depth backbone others distill from.

This page covers the mixing recipe. Single-image basics are monocular depth estimation; the transformer upgrade is DPT; the larger successor is Depth Anything.

Think of It Like This

Blending cookbooks with different cup sizes

Five cookbooks use different cup sizes, so raw cup counts never match. Convert each recipe to ratios first, then blend: the shared ratios survive while cup sizes wash out.

Datasets are cookbooks and median alignment is the ratio conversion. The analogy stops at content: recipes share food while depth mixes indoor rooms with driving streets, which only works because geometry cues overlap.

How It Actually Works

Each training sample predicts inverse depth $\hat{d}$ ; per-sample least squares fits scale $s$ and shift $t$ to truth $d$ , and the loss scores $|s \hat{d} + t - d|$ trimmed at the largest $20$ percent of residuals. Multi-objective mixing samples each dataset round-robin so large sets cannot drown small ones. Version $3$ swaps the CNN for a DPT vision-transformer backbone with a convolutional decoder, raising resolution to $384$ and sharpening edges while keeping the same invariant loss.

Worked numbers

Two samples with truths $d = (1.0, 2.0)$ and raw outputs $(0.4, 0.9)$ fit $s \approx 2.2$ , $t \approx 0.1$ , giving aligned $(0.98, 2.08)$ with mean error $0.05$ . Trimming the top $20$ percent drops a $1.5$ outlier pixel from the mean, cutting batch loss from $0.18$ to $0.09$ in the demo split.

Code

1def align_scale_shift(pred: list[float], gt: list[float]) -> tuple[float, float]:2    mean_p = sum(pred) / len(pred)  # crude least squares stand-in3    mean_g = sum(gt) / len(gt)4    scale = sum((p - mean_p) * (g - mean_g) for p, g in zip(pred, gt)) / sum((p - mean_p) ** 2 for p in pred)5    return scale, mean_g - scale * mean_p6
7
8print([round(v, 2) for v in align_scale_shift([0.4, 0.9], [1.0, 2.0])])  # -> [2.0, 0.2]

Watch Out For

Using MiDaS numbers as meters

MiDaS outputs disparity up to an unknown affine map, so raw values drift per image. The symptom is great ordering with wrong distances. Median-align to sparse truth or a known height before any metric use.

The Quick Version

MiDaS mixes many depth datasets with a scale-and-shift-invariant loss.
Per-sample alignment lets incompatible units train one model.
Trimmed losses ignore the worst residuals such as sky and mirrors.
Version $3$ pairs a transformer backbone with a convolutional decoder.
Output is relative disparity, never meters without alignment.
It is the standard teacher for later foundation depth models.