Occupancy Networks

Occupancy networks predict whether each 3D cell around the car is free, occupied, or unknown, giving planning a dense safety volume.

Space around the car is voxelized into free, occupied, and unknown cells for planning

Why Does This Exist?

Object lists miss what they were never trained to name: a fallen ladder, a hanging tarp, an open car door. Planning still must not drive through them. Occupancy networks exist to represent everything geometrically: divide space into voxels and label each one free, occupied, or unknown, regardless of class. Unknown matters as much as occupied because occluded space must stay suspect until observed.

For flat top-down maps see BEV perception and for geometry see stereo vision.

Think of It Like This

A ice-cube tray with three states

You hold a tray of cubes around the car. Each cube is clear for free, solid for blocked, or frosted for unseen. You plan through clear cubes and never through frosted ones. New views defrost cubes one by one. The analogy stops at ice: voxels also carry semantics and motion when available.

How It Actually Works

Ego space such as $80 \times 80 \times 8$ m is voxelized at 0.5 m into $160 \times 160 \times 16$ cells, about 409,600 voxels. Image and lidar features lift into the volume, and a 3D head predicts free, occupied, or unknown per voxel plus optional semantics. Ray casting from lidar marks observed free space along each return, distinguishing unknown behind surfaces from confirmed free. Planning queries the grid for collisions along candidate paths.

Worked example

A ladder lying across the lane occupies about $12 \times 2 \times 1$ voxels at 0.5 m resolution, 24 cells no detector names. The occupancy head marks 20 of them occupied with 0.85 mean confidence, enough to invalidate the straight path. Occluded cells behind a truck stay unknown rather than free, so the planner slows instead of assuming clearance. The object list stays empty while the volume still stops the car.

Code

This snippet marks free space along a ray.

1import numpy as np2
3
4def mark_ray(grid: np.ndarray, length: int) -> np.ndarray:5    g = grid.copy()6    g[:length] = 0  # 0 free along observed ray7    g[length] = 1  # 1 occupied at return8    return g9
10
11ray = np.full(10, 2)  # 2 unknown12print(mark_ray(ray, 6).tolist())  # -> [0, 0, 0, 0, 0, 0, 1, 2, 2, 2]

Cells past the return stay unknown. Planning must treat 2 as blocked until seen.

Watch Out For

Unknown treated as free for comfort

Thresholding unknown into free removes phantom braking and adds real collisions in occluded intersections. The symptom is smooth demos with near misses at parked-truck crossings. Keep the three-state output into planning, and let the planner price unknown explicitly with slowdown rather than optimism.

The Quick Version

Occupancy grids label every voxel free, occupied, or unknown.
Geometry catches unnamed obstacles that detectors miss.
Unknown is a first-class state for occluded space.
Resolution and range trade memory against safety margin.
Planning queries volumes, not just object lists.