BEVFormer Camera 3D Detection

BEVFormer turns six camera views into one bird's-eye map by letting each map cell ask the right image pixels for evidence.

BEVFormer lifts six camera views into one bird's-eye grid with spatial and temporal attention

Why Does This Exist?

Cameras are cheap and dense but each view is flat. Lifting six surround views into 3D by depth guessing smears far objects. Stereo vision needs overlap, which surround cameras lack. BEVFormer (2022) avoids explicit depth: it keeps a learnable bird's-eye grid and lets each grid cell pull evidence from the image views through attention, with past grids carried forward for motion.

This page covers the BEV query mechanism. Pure LiDAR voxel methods like SECOND are the alternative when depth accuracy matters most.

Think of It Like This

Six security monitors and one floor plan

Six guards watch six monitors while you hold a floor plan of the shop. You point at one square of the plan and ask each guard only about the pixels that show that square. Then you update the plan square and move to the next one, remembering last minute's plan too.

BEV cells are floor squares, cameras are monitors, and attention is the pointing. The analogy stops at geometry: the pointing uses calibrated projection, not hand waving.

How It Actually Works

The BEV grid has $H = 200$ by $W = 200$ queries covering $100$ m, so each cell spans $0.5$ m. Each query first attends to the previous BEV grid (temporal self-attention), then samples $K = 4$ points per camera view through deformable cross-attention. Projection $P$ maps a 3D reference point $(x, y, z)$ to pixel $(u, v)$ per camera; points outside a view are skipped.

Worked example

A query at cell $(100, 120)$ means $(0.0, 10.0)$ m in ego coordinates. With reference heights $z \in \{-1, 0, 2\}$ m, the front camera projection lands near $(u, v) = (812, 430)$ , $(810, 470)$ , $(808, 540)$ in a $1600 \times 900$ image. The query samples 4 offsets around each, weights them $[0.5, 0.3, 0.15, 0.05]$ , and updates its feature from $[0.2, 0.1]$ to $[0.62, 0.44]$ . After the head, the cell emits car score $0.83$ with box center $(0.1, 10.2, 0.8)$ m.

Watch Out For

Stale history smearing fast motion

Carrying the full past BEV grid with ego-motion error above $0.3$ m drags boxes behind moving cars. The symptom is velocity underestimated by $2$ m/s. Re-align history with exact ego pose before temporal attention.

Reference heights missing low curbs

Sampling only $z \in \{0, 2\}$ m skips a $0.15$ m curb entirely. Add a $-1$ m reference so ground-level evidence reaches the query.

The Quick Version

BEVFormer keeps one bird's-eye query grid fed by all cameras through attention.
Temporal attention carries the past grid forward for velocity and occlusion.
No explicit depth map is built; projection picks the pixels per query.
Do not use it where centimeter depth matters; LiDAR still wins there.
A $200 \times 200$ grid at $0.5$ m cells covers highway range without per-pixel depth.