BEVDet Camera to BEV 3D Detection

BEVDet predicts per-pixel depth, splats image features into bird's-eye view, then detects 3D boxes there.

Why Does This Exist?

Camera features live in perspective view while driving decisions live in bird's-eye view. BEVFormer pulls image evidence with attention, which is accurate but heavy. BEVDet (2022) takes the explicit route: predict depth per pixel, lift each feature along its ray into 3D, splat into a BEV grid, and detect there. It reuses image backbones and a CenterPoint-style head, so teams get a simple camera-only baseline.

This page covers lift-splat-detect. When calibration drifts or night rain ruins depth, the attention route in BEVFormer degrades more gracefully.

Think of It Like This

Stacking transparencies by guessed distance

You hold six photos and a glass box marked in meters. For each photo pixel you guess its distance, walk that far into the box, and drop a dab of paint. After all pixels, you look down through the box and draw boxes around the paint piles.

Depth bins are the meter marks and splatting is the paint. The analogy stops at learning: distances are predicted and supervised, not eyeballed.

How It Actually Works

Each image feature $F(u, v)$ of dim $C = 64$ gets a depth distribution $D(u, v)$ over $B = 80$ bins from $2$ m to $60$ m. The lifted point is $D_b \cdot F$ placed at 3D position from intrinsics and extrinsics. All lifted points falling in BEV cell $(i, j)$ sum-pool into that cell. A BEV encoder then smooths the grid before the center head predicts heat, size, and yaw.

Worked example

A front pixel at $(800, 450)$ has depth weights $[0.05, 0.70, 0.20, 0.05]$ over bins $\{9, 10, 11, 12\}$ m. Its feature $[0.4, 0.2]$ lifts mostly to $10$ m: contribution $0.70 \times [0.4, 0.2] = [0.28, 0.14]$ lands in cell $(100, 120)$ for $(0, 10)$ m. After pooling 40 such pixels the cell holds $[3.1, 1.6]$ , the head predicts car peak $0.79$ with center $(0.1, 10.1)$ m and yaw $0.02$ rad.

Watch Out For

Depth bins too coarse past 40 meters

With $1$ m bins past $40$ m, a car at $45.4$ m smears across two cells and the center splits in half. The symptom is doubled boxes at range. Use finer bins near and accept coarser far, or switch to attention lifting.

Splat overflow on the ego vehicle

Pixels showing the ego car's own hood splat into the center cells and fake a permanent obstacle. Mask the ego mask region in every view before lifting.

The Quick Version

BEVDet predicts depth per pixel and splats features into bird's-eye view.
Detection then runs on the BEV grid with a center-style head.
It is simpler and faster to train than attention lifting.
Do not use it when depth supervision is missing; errors compound into the grid.
Eighty bins from $2$ m to $60$ m cover city and highway with one model.