Visuomotor Policy Learning

Visuomotor policies map images straight to joint motions, learning contact skills from demonstrations and refining them with trial feedback.

Images condition an action sampler that outputs smooth joint motions refined by feedback

Why Does This Exist?

Staged primitives cover rigid pick-and-place but strain on wiping, folding, or tool use where contact dynamics resist hand rules. Visuomotor policies exist to learn those skills from demonstrations: map images to action sequences directly, capture multimodality such as two valid grasps, and smooth motions without waypoint engineering. Sim-to-real transfer for these policies is owned by the reinforcement-learning plan and linked there rather than re-taught here.

For staged control see the manipulation page in this wave and for tracking see object tracking.

Think of It Like This

Learning tennis by rallying, not by physics

You rally a thousand balls and your arm finds topspin without solving equations. Demos are the rallies, the policy is the arm. The analogy stops at courts: the robot also needs safety bounds because it cannot feel pain when it errs.

How It Actually Works

Observations $o$ such as wrist and scene images plus proprioception condition a policy $\pi(a \mid o)$ over action chunks. Behavior cloning fits demo actions directly. Diffusion or transformer variants sample multimodal action distributions instead of averaging two valid moves into one invalid middle. Rollouts refine with corrections or reinforcement signals. Action chunks of 8 to 16 steps at 10 Hz smooth contact while replanning fast enough to react.

Worked example

A wiping task offers left-to-right and right-to-left strokes, both valid. Mean regression averages them into staying still with 0.50 weight each and zero motion. A diffusion policy keeps both modes and samples one per rollout, completing the wipe in 12 of 15 trials versus 2 of 15 for the averaged policy. Multimodality is the measured gap, not a preference.

Code

This snippet shows why averaging modes fails.

1import numpy as np2
3
4def mean_vs_sample(modes: list[float]) -> tuple[float, float]:5    mean = float(np.mean(modes))6    sample = float(np.random.default_rng(0).choice(modes))7    return round(mean, 2), round(sample, 2)8
9
10print(mean_vs_sample([-1.0, 1.0]))  # -> (0.0, 1.0)

Mean 0.0 stays still while sampling commits to a real stroke. Policies must sample, not average.

Watch Out For

Demo-only policies outside the demo envelope

Lighting shifts or a moved bowl put observations off-manifold and actions degrade sharply. The symptom is smooth lab demos with sudden workspace failures. The fix bounds the envelope, adds corrective demos around failures, and gates deployment with out-of-distribution checks rather than blind trust.

The Quick Version

Visuomotor maps images plus state to action chunks.
Demonstrations teach contact skills that rules miss.
Multimodal samplers beat mean regression on real tasks.
Chunks smooth motion while replanning stays reactive.
Sim-to-real transfer links to its RL owner; this page owns the policy.