Long-Horizon Planning for AI Agents

A reactive agent picks the best next step and walks into dead ends. Long-horizon planning writes milestones first, checkpoints each one, and replans the rest when a check fails.

Three milestones of four steps turn a 12-step job into checkpointed runs, keeping 2300 tokens instead of 6000 and bounding a crash to four steps.

Why Does This Exist?

A reactive agent reads the current state and picks the best next action. That wins short jobs: answer one question, sort one list, book one flight. Stretch the job to an afternoon and the same habit walks into dead ends. It books the cheapest hotel first, then learns the conference venue moved across town. It migrates forty files, then learns step two picked the wrong schema. Each step looked good alone. Together they waste the run.

Long-horizon planning exists to bound that waste. The agent writes milestones before acting, saves state after each one, and revises the remaining plan when a check fails. A crash then costs one milestone, not the whole job, and context stays flat instead of growing with every step.

This page covers milestones, checkpoints, and the replan check. The general step machinery lives in planning in agents. Breaking one goal into ordered pieces lives in task decomposition. Splitting work across specialists lives in multi-agent orchestration. What the loop remembers between milestones lives in memory systems in agents.

Basics first: a tool is a function the model may name with arguments, such as searching flights or reading a file. The model only writes the name and the arguments. Your code runs the function and pastes the result back. If that handoff is new, read ReAct first, because every plan below assumes that loop.

Think of It Like This

Crossing a city with waypoints, not one staring contest

You must cross an unfamiliar city on foot with a heavy bag. The reactive move is to stare at the destination and walk toward it. You gain ground fast, then hit a river with no bridge and retrace an hour.

The planning move is to mark three waypoints on a paper map: the market, the bridge, the square. You walk to the market, drink water, and confirm the bridge is open. If it is closed, you redraw only the rest of the route from where you stand. You never re-walk the finished part, and you never carry the whole city in your head at once.

Where it stops: your map stays still while you walk. An agent's world model drifts with every simulated step, so a waypoint twelve steps out can be fiction. Paper never lies about a bridge. A simulated future often does.

How It Actually Works

Call the goal $G$ , which means the distant outcome you want. Call a milestone a chunk of steps with its own check, such as four setup steps ending in a tested login. Call $M$ the milestone count, $k$ the steps per milestone, and $N$ the total steps, so $N = M \times k$ . Call $S_t$ the saved state after step $t$ : the plan, the summaries so far, and the tool receipts such as file paths and IDs. Call $C_t$ the context fed to the model at step $t$ .

Milestones turn one gamble into checked bets

The planner splits $G$ into $M$ milestones before acting. Each milestone ends with a check written in advance: four files migrated and tests passing, not just four files touched. The executor works one milestone at a time. When a check fails, the planner revises only the remaining milestones from the current $S_t$ . Finished milestones never rerun.

Checkpoints bound how much a crash costs

A checkpoint writes $S_t$ to durable storage after each milestone. Without one, a timeout at step 11 wastes 11 steps. With one every $k = 4$ steps, a crash wastes at most 4. The same write bounds context. Raw history grows with $N$ . Kept context after compaction is $M$ summaries plus the last milestone raw, which stays flat while $N$ grows. The full loop that runs each step lives in planning in agents, and the stores that hold $S_t$ across sessions live in memory systems in agents.

Worked example: 3 milestones, 12 steps

Take $M = 3$ milestones with $k = 4$ steps each. Then $N = 3 \times 4 = 12$ steps. Say each raw step costs 500 tokens and each milestone summary costs 100 tokens. Raw history would be $12 \times 500 = 6000$ tokens. Kept context is $3 \times 100 + 4 \times 500 = 300 + 2000 = 2300$ tokens. The saving is $6000 - 2300 = 3700$ tokens, and the diagram draws exactly this split.

Reliability follows the same shape. Write per-step success as $q = 0.97$ , meaning each step finishes without a timeout or bad tool reply. Assume steps fail independently. The chance all 12 finish in one go is $0.97^{12} \approx 0.694$ , so about 69 runs in 100 finish cleanly. Without checkpoints the other 31 waste up to 12 steps each. With checkpoints every 4 steps, a crash wastes at most 4. That bound is the whole purchase.

Reach for milestones when the job runs past about 10 steps, touches outside systems, or must survive restarts. Skip them for a single lookup, where one plan plus one checkpoint costs more than the job itself.

Code

The point here is the budget check. This stdlib-only sketch recomputes the step count, the token fit, and the one-go success chance from the worked example:

1def kept_tokens(milestones: int, per_summary: int, raw_steps: int, per_step: int) -> int:2    return milestones * per_summary + raw_steps * per_step3
4
5M, K = 3, 46N = M * K7print(N)  # -> 128print(N * 500)  # -> 60009print(kept_tokens(M, 100, K, 500))  # -> 230010print(round(0.97**N, 3))  # -> 0.694

Twelve steps, 6000 raw tokens against 2300 kept, and a 0.694 one-go chance. Change $M$ to 6 and watch kept context stay near 2600 while raw doubles to 12000.

Watch Out For

Trusting a simulated state ten steps out

The symptom is a confident plan that dies on contact: the agent predicted a file, an ID, or a price five steps ahead, chained three more steps onto that guess, and the first real observation matches none of it. The run then replans from fiction, not from the observation.

Fix it by grounding early and often. Keep milestones short, check each one against a real observation, and never let a plan chain more than a few simulated states without a tool reply. Store receipts verbatim in $S_t$ and treat anything else as a guess.

Planning with tools you never built

The symptom is a beautiful milestone list that fails on step one: reserve a room via the room API, when no room tool exists. The planner wrote what sounded logical, not what your toolbox contains.

Fix it by pasting the tool list with one-line descriptions into the planner prompt, and by rejecting any step that names an unknown tool before executing anything. The step loop that enforces this lives in ReAct.

The Quick Version

Long-horizon planning splits a distant goal into milestones with checks, checkpoints each one, and replans the rest on failure.
Three milestones of four steps make 12 steps: 6000 raw tokens shrink to 2300 kept, saving 3700.
At 0.97 per-step success, all 12 steps finish in one go about 69 times in 100 tries.
A crash without checkpoints wastes up to 12 steps, while checkpoints bound the loss to 4.
Milestones pay off past about 10 steps or when restarts are likely, not for single lookups.
Never chain many simulated states without a real observation, and never plan with tools you did not build.