3D Occupancy Prediction for Driving
Occupancy prediction marks every bird's-eye cell as free, car, or pedestrian so planners see around occlusions.
Why Does This Exist?
Boxes miss unknown shapes like fallen cargo and overhangs. Occupancy fields give the representation; driving needs it predicted from sensors each frame, including occluded cells behind buses. This page covers the prediction head that serves planning directly.
Think of It Like This
A fog-of-war map that guesses behind hills
Strategy games reveal terrain near units and guess hills beyond from contours. Commanders route around guessed hills rather than driving blind. Guesses update as scouts advance.
Sensors are scouts and grids are the map. The analogy stops at classes: games use friend or foe while driving uses labels.
How It Actually Works
A BEV encoder outputs cells at m covering m. Each cell predicts classes (free plus objects). Loss weights occluded cells from ray casting and small classes -fold. Planning reads free probability as drivable.
Worked example
A bus hides cells of crosswalk. Visible road cells score free . Hidden cells behind the bus predict pedestrian , free , car , so the planner slows though no box exists. After the bus passes, the same cells read free . The early pedestrian mass is the safety gain over boxes.
Watch Out For
Unknown cells planned as free at speed
Thresholding unknown as free at m/s leaves no braking room. The symptom is late braking at occluded crosswalks. Cap speed by unknown mass ahead, not just visible free space.
The Quick Version
- Occupancy labels every cell, visible or hidden, for planners.
- Seventeen classes cover free space plus road users and props.
- Occluded cells train from ray-cast supervision, not boxes.
- Do not plan unknown as free at speed; slow for hidden mass.
- A grid at m serves highway decisions.