Object Manipulation

Object manipulation turns grasps into multi-step tasks like pick, place, and insert, sequencing vision, motion, and contact checks.

Perceive, grasp, transport, and place form a staged loop with checks between steps

Why Does This Exist?

Grasping lifts the object. Manipulation finishes the job: carry without collision, align to a target within millimeters, insert without jamming, and release cleanly. Each step fails differently, and blindly chaining open-loop motions compounds error into broken parts. Object manipulation exists to stage the task into checkable primitives with vision and force gates between them.

For grasps see the grasp page in this wave and for poses see pose estimation.

Think of It Like This

Moving a cake across a busy kitchen

You spot the cake, grip the tray level, walk without tilting, align over the stand, and set down gently, checking at each handoff. Skip a check and frosting meets the floor. Robot manipulation stages the same checks. The analogy stops at cake: robots also quantify clearance and force thresholds numerically.

How It Actually Works

A task decomposes into perceive, grasp, transport, align, insert or place, and release. Each primitive takes an image or pose goal plus a success predicate: grasp confirmed by force, transport clear by voxel check, alignment within tolerance by vision, insertion by force drop. Failure triggers local recovery such as re-grasp or re-align rather than restarting the task. Tolerances tighten along the chain from centimeters in transport to sub-millimeter in insertion.

Worked example

A peg 10 mm wide enters a 10.5 mm hole with 0.25 mm clearance per side. Transport holds 5 mm tolerance, visual alignment refines to 0.5 mm, and compliant insertion with force feedback completes the final 0.25 mm. Skipping alignment and pushing from 5 mm offset wedges the peg with a force spike, which the monitor aborts. The staged tolerances are the mechanism, not overhead.

Watch Out For

Tolerances that never meet

Transport promises 5 mm while insertion needs 0.25 mm, but no alignment step bridges them. The symptom is repeated jamming at the same depth. The fix inserts an explicit fine-alignment primitive with its own sensor and predicate, and proves the tolerance chain on paper before running hardware.

The Quick Version

Manipulation is staged primitives with checks, not one motion.
Tolerances tighten from transport to insertion.
Vision plus force predicates gate each handoff.
Local recovery beats full restarts on failure.
Prove the tolerance chain before touching parts.