Trajectory Prediction

Trajectory prediction forecasts where each nearby road user will be over the next few seconds, as several possible paths with probabilities.

One observed track fans into three predicted paths whose probabilities sum to one

Why Does This Exist?

Planning cannot brake for where a pedestrian is; it must brake for where the pedestrian will be. A single straight-line guess fails at intersections where turning, crossing, or waiting are all plausible. Trajectory prediction exists to produce that future as a small set of paths with probabilities, so the planner can stay safe under the likely ones without freezing for every remote possibility.

Tracks come from multi-object tracking and identity scoring from tracking metrics.

Think of It Like This

Calling the next move at a crosswalk

You watch a stranger approach a crosswalk with a phone in hand. You hold three futures at once: they cross, they wait, they turn along the sidewalk. Your feet prepare for the crossing one while your eyes keep the others. The model holds the same three paths with numbers attached. The analogy stops at minds: the model reads motion and map, not intention.

How It Actually Works

History $H$ of past positions plus map $M$ of lanes and crosswalks maps to $K$ future paths $Y_1 \dots Y_K$ with probabilities $p_1 \dots p_K$ summing to 1. Here $K$ is small, often 3 to 6. Encoders read agent motion and nearby map polylines. Interaction layers let agents influence each other, so a braking lead car bends the follower forecast. Decoding outputs waypoints every 0.5 s out to 3 to 5 s. Training scores the closest mode hardest, keeping diversity instead of collapsing to the mean.

Worked example

A cyclist approaches at 5 m per s, 15 m from a junction. Modes read straight 0.55, right turn 0.30, stop 0.15, summing to 1.0. After 3 s the straight endpoint sits 15 m ahead, the turn endpoint 8 m ahead plus 6 m right, the stop endpoint 3 m ahead. If truth goes straight, the best-mode error is under 1 m while the mean of all modes errs by 4 m. That gap is why multimodal output beats single-path regression.

Code

This snippet keeps probabilities honest.

1import numpy as np2
3
4def normalize(p: list[float]) -> list[float]:5    a = np.array(p, dtype=float)6    return (a / a.sum()).round(3).tolist()7
8
9print(normalize([0.55, 0.30, 0.15]))  # -> [0.55, 0.3, 0.15]10print(sum(normalize([2.0, 1.0, 1.0])))  # -> 1.0

Modes must sum to one before planning trusts them. Renormalize after any pruning.

Watch Out For

Mode collapse to the average path

Training all modes toward the single truth pulls them into one blurry middle that drives straight through curbs. The symptom is low single-mode error with absurd paths at junctions. Score only the best mode plus a diversity term, and inspect endpoints on turning cases specifically.

The Quick Version

Prediction outputs several futures with probabilities, not one line.
Map and neighbor interaction shape which modes survive.
Horizons run 3 to 5 seconds at half-second steps.
Best-mode error judges quality; mean error punishes honest uncertainty.
Planning consumes the set, keeping safety without freezing.