VoxelNet Voxel 3D Detector Model

Chop space into small cubes, learn features per cube, then detect in 3D. VoxelNet proved end-to-end voxels beat hand lidar features.

Points group into voxels that sparse convolutions turn into 3D boxes

Why Does This Exist?

Early lidar detectors hand-built occupancy and height maps, losing shape detail. VoxelNet exists to learn the voxel features instead: a small PointNet inside each voxel plus sparse 3D convolutions plus a bird's-eye-view head, trained end to end. It set the voxel path that SECOND sped up and PointPillars flattened.

Context is LiDAR based sensing and 3D object detection.

Think of It Like This

Ice cube trays with smart molds

Freeze rain into ice trays where each cube learns its own crystal shape, then stack trays into a freezer map of the storm. Hand trays would just count water.

Voxels are trays and VFE layers are the smart molds. The analogy stops at emptiness: most trays stay empty, which is why sparse math matters.

How It Actually Works

Space voxels at $0.2$ m keep up to $35$ points each; a voxel feature encoder runs shared MLPs plus max-pool per voxel into a $128$ -dim code. Sparse $3 \times 3 \times 3$ convolutions process only nonempty voxels, downsampling threefold before reshaping height into bird's-eye-view maps. An RPN head regresses $(x, y, z, l, w, h, \theta)$ residuals against anchors with $0.7$ car IoU matching.

Worked numbers

A car holds about $800$ points across roughly $120$ voxels, about $7$ points per voxel after sampling. Empty voxels exceed $95$ percent of the grid, so sparse math skips nineteen of twenty positions. Reference KITTI moderate car AP lands near $81$ percent at about $4$ frames per second on early GPUs.

Code

1def voxel_index(x: float, size: float) -> int:2    return int(x // size)  # quantize metric position to grid3
4
5print(voxel_index(4.05, 0.2))  # -> 20

Watch Out For

Voxelizing too coarsely for pedestrians

A $0.4$ m voxel swallows a $0.6$ m pedestrian into two cells, erasing limbs. The symptom is great car AP with collapsed people scores. Keep voxels near $0.2$ m or finer for small classes, and budget the extra sparse cost.

The Quick Version

VoxelNet learns per-voxel features with a mini-PointNet encoder.
Sparse 3D convolutions skip the $95$ percent empty grid.
Height folds into bird's-eye-view maps for the detection head.
Anchors with $0.7$ IoU matching train box plus heading regression.
Reference KITTI car AP sits near $81$ percent moderate.
Fine voxels matter most for pedestrians.