Robot Task and Motion Planning

Picking the right sequence of actions isn't enough if no physical motion can execute it. Task and motion planning searches action choices together with grasps, poses and collision-free movements that make those choices possible.

A payload that cannot pass broadside through an opening fits after a quarter-turn, changing which action plan is feasible.

Why Does This Exist?

“Pick the box, then place it inside” can be logically sensible and physically impossible with the selected grasp. The held box might be too wide for the opening, or the arm might have no collision-free route while carrying it. Task and motion planning, or TAMP, connects the action sequence with the continuous physical choices needed to execute it.

A task planner reasons about facts such as whether a hand is empty. A motion planner reasons about configurations and collision-free paths. Grasp targets supply candidate contacts; motion planning supplies geometric connections. TAMP must let failures in those connections change the action choices, rather than treating geometric execution as a final afterthought.

Think of It Like This

Carrying a table through a doorway

Your first plan is to lift the table and carry it through the door. Checking the doorway reveals that the table won't fit broadside. You can turn it before entering, remove an obstruction, or choose another route. Each choice changes both the sequence of actions and the shapes that must fit during movement.

The action list is the task plan; the carried orientation and doorway clearance are motion constraints; the successful continuous movement is the plan's physical witness. The limit is sensing: a person adjusts while carrying, whereas a deterministic planner needs an explicit model and execution feedback to handle an unexpected chair or an inaccurate doorway width.

How It Actually Works

Give actions preconditions and effects

Represent a symbolic state as a set of facts. A simplified Pick(box, grasp) requires HandEmpty and At(box, source), together with appropriate geometric conditions. Its effects remove HandEmpty and At(box, source) and add Holding(box, grasp). A Place action requires the holding relation and a valid destination; it restores HandEmpty and adds the destination fact.

These facts are abstractions, not observations that magically become true when requested. A pick also needs a reachable grasp, a clear approach and a suitable contact model. The holding relation attaches the object's geometry to the tool: subsequent collision checks must include that object. The integrated TAMP survey, Sections 2–3, describes the discrete action structure and continuous constraints, including changing attachments.

Bind the action skeleton to geometric witnesses

An action skeleton fixes a sequence such as move, pick, carry, place, while leaving values open. Let $\pi$ name that sequence and $y$ collect its continuous parameters: grasps, placement poses, joint configurations and paths. A compact statement of the planning problem is

\text{find }(\pi,y)\quad\text{such that}\quad \text{Preconditions}(\pi,y)\ \text{hold and}\ C_i(\pi,y)\ \text{holds for every }i.

$C_i$ is a physical constraint, such as inverse-kinematic reachability, non-collision throughout a motion, or a stable placement. A witness is an actual parameter assignment that passes its stated check, not merely a plausible action name.

PDDLStream, Section 3, represents continuous procedures as conditional generators with facts they certify about their outputs. A grasp generator may return a pose; an inverse-kinematics routine may return joints; a motion planner may return a path. Optimistic planning can consider a candidate before computing every witness, but execution requires resolving the needed checks. A planner should not interpret an optimistic placeholder as a measured grasp or an executable trajectory.

Worked example: an opening constrains the plan

Consider a hypothetical top-down rectangle with sides $a=0.14$ m and $b=0.08$ m. It translates straight through a parallel-sided opening of clear width $w=0.12$ m while holding constant yaw $\theta$ . The rectangle's cross-passage extent is

W(\theta)=a|\cos\theta|+b|\sin\theta|.

This follows by projecting each side onto the passage-width axis and adding its extent. Assume the rectangle is centered in the passage, the other direction is unbounded, and touching counts as collision, so require $W<w$ .

At $\theta=0$ , $W=0.14$ m and $w-W=-0.02$ m: the broadside placement overlaps the walls. At $\theta=\pi/2$ , $W=0.08$ m and $w-W=0.04$ m, leaving $0.02$ m on each side. The two diagram panels use the same scale and opening width. Rotating the rectangular payload swaps its dimensions; the broadside overlaps are intentional collision geometry.

Now give three finite candidate plans hypothetical action costs. A direct pick/place plan has cost 2 but fails the aperture check. An orient/pick/place plan has cost 3 and passes. Relocating a movable obstruction before pick/place has cost 4 and opens a $0.18$ m gap, so the broadside transfer passes. The least-cost feasible member of this tiny candidate set is the cost-3 plan. These integer costs are assumptions about actions, not measured execution times or a claim of global optimality.

The check is exact only for the declared straight, constant-yaw rectangular transfer. Turning beside the doorway, gripper clearance, vertical geometry, arm reachability, force balance and approach/retreat still need witnesses. A smaller width proves one necessary condition in a full robot task, not all of feasibility.

Which failure should send the search backward?

A fixed grasp that cannot pass an exact aperture test can be rejected for that geometry. A motion planner timing out on a sampled grasp is weaker evidence: another path, grasp or sampling budget might succeed. Preserve that distinction when caching failures. Cache geometry-dependent results using the relevant scene, attachment and parameter values; moving an object can invalidate a previously clear path.

Joint search costs more than a fixed scripted sequence because geometric checks can be expensive and failures cause branching. Use it when grasp, placement, route or action order genuinely affect each other's feasibility. A repetitive, fixtured task with one verified sequence may need an execution state machine and local recovery rather than general TAMP. Deterministic TAMP also assumes its state model is adequate; observations and replanning are needed when execution changes that state unexpectedly. Learned proposals can guide search, but still require physical checks under the chosen model.

Code

This standalone finite enumeration makes the logical/geometric split inspectable. It is not an implementation of PDDLStream or a general TAMP solver.

1from dataclasses import dataclass2from math import cos, isclose, pi, sin3
4@dataclass(frozen=True)5class Plan:6    name: str7    yaw: float8    gap: float9    cost: int10
11def width(yaw: float, a: float = 0.14, b: float = 0.08) -> float:12    return a*abs(cos(yaw)) + b*abs(sin(yaw))13
14plans = [Plan("direct", 0.0, 0.12, 2),15         Plan("orient first", pi/2, 0.12, 3),16         Plan("relocate obstruction", 0.0, 0.18, 4)]17feasible = [p for p in plans if width(p.yaw) < p.gap]18chosen = min(feasible, key=lambda p: p.cost)19assert isclose(width(0), 0.14)20assert isclose(width(pi/2), 0.08)21assert plans[0] not in feasible22# Touching is forbidden in this exact teaching model.23assert not width(0) < 0.1424for p in plans:25    print(f"{p.name}: total gap={p.gap-width(p.yaw):.3f} m")26print(f"chosen: {chosen.name}, cost {chosen.cost}")

Output: direct: total gap=-0.020 m, orient first: total gap=0.040 m, relocate obstruction: total gap=0.040 m, and chosen: orient first, cost 3.

Watch Out For

An empty-hand path is reused after picking up a load

The arm clears the opening while the carried box hits its frame. Attach the payload collision model at the grasp transform and recheck the swept geometry, including self-collision. A task-level Holding fact must change the geometric state used by the planner.

One failed sample becomes a universal impossibility claim

A solver rejects an entire pick action after one IK seed or motion attempt fails. Record whether the result is a proof for fixed parameters, a sampled failure or a timeout. Backtrack to other bindings and action skeletons without permanently banning a feasible action on insufficient evidence.

The Quick Version

TAMP couples discrete action choices with continuous grasps, poses and paths that witness feasibility.
Preconditions and effects describe state changes; attached payloads also change collision geometry.
The 140 by 80 mm fixture passes a 120 mm opening only in the checked narrow orientation among the two shown cases.
A sampled planning failure, an optimistic candidate and a verified executable plan are different kinds of evidence.