CenterPoint 3D Detection by Centers

CenterPoint finds 3D objects by predicting their centers on a bird's-eye map instead of placing thousands of anchors.

CenterPoint predicts center heat peaks on a bird's-eye map and reads each 3D box from its peak

Why Does This Exist?

Anchor-based LiDAR detectors tile every position with several pre-sized boxes and then classify each one. Most anchors are empty, and small size errors hurt. Anchor boxes work in 2D but in 3D the anchor count explodes across yaw and height. Tracking then needs a second association step.

CenterPoint (2021) drops anchors entirely. It predicts a heatmap of object centers on the bird's-eye view and regresses size, height, and yaw only at the peaks. The same centers feed velocity regression, so detection and tracking share one representation. This page covers the center head. The voxel backbone underneath can be a SECOND-style encoder.

Think of It Like This

Finding tents by their campfires at night

From a hilltop you cannot see tent outlines in the dark, but each tent has one campfire. You mark every flame on your map, walk to each flame, and then measure the tent around it. No flame, no tent.

Centers are campfires and boxes are tents. The heatmap marks flames, and regression measures the tent. The analogy stops at motion: campfires do not carry velocity, but CenterPoint peaks do.

How It Actually Works

The bird's-eye feature map has stride $s = 0.075$ m per cell. Ground-truth centers are splatted into a heatmap $Y$ with a Gaussian of radius $r = 2$ cells. The network predicts $\hat{Y}$ plus box attributes at each cell: offset $(\delta x, \delta y)$ , height $z$ , size $(l, w, h)$ , yaw $(\sin\theta, \cos\theta)$ , and velocity $(v_x, v_y)$ .

Worked example

A car center at $(10.0, 5.0)$ m maps to cell $(133, 66)$ since $10.0/0.075 = 133.3$ and $5.0/0.075 = 66.7$ . The target heat at the peak cell is $1.0$ , at one cell away $\exp(-1/2) \approx 0.61$ . The model predicts peak $0.88$ with offset $(0.3, 0.7)$ cells, which corrects to $(10.02, 5.05)$ m, size $(4.5, 1.9, 1.6)$ m, yaw $0.05$ rad, velocity $(6.0, 0.2)$ m/s. A $0.05$ m center error keeps IoU above $0.8$ .

Watch Out For

Heatmap threshold hiding distant pedestrians

A fixed peak threshold of $0.5$ keeps cars but deletes a pedestrian peak at $0.32$ past $30$ m. The symptom is recall that falls with range. Lower the threshold to $0.2$ for small classes and let the box scorer filter.

Yaw flip on symmetric cars

Predicting $\theta$ directly flips a car by $\pi$ because front and back look alike. The fix is the $\sin\theta$ and $\cos\theta$ pair, which makes $0$ and $\pi$ far apart in target space.

The Quick Version

CenterPoint predicts center heat peaks instead of scoring anchor boxes.
Size, yaw, height, and velocity are read only at the peaks.
One representation serves detection and tracking together.
Do not use it when objects overlap heavily in bird's-eye view; peaks merge.
Gaussian radius $2$ cells balances localization against missed small objects.