Active Vision

A fixed camera accepts whatever the scene offers. Active vision moves the camera — zooming, refocusing, circling — to collect the views that answer the question cheapest.

The zoom view wins at 0.27 bits per cost against 0.12 for panning, pricing information like freight

Why Does This Exist?

Fixed viewpoints leave questions permanently open: the mug handle hides behind the body, the license plate stays sub-pixel, the defect sits in shadow. No classifier, however large, recovers pixels the camera never captured. Passive perception treats missing information as fate.

Active vision treats it as a shopping list. Pan, tilt, zoom, focus, and base motion become perception actions chosen to maximize expected uncertainty reduction per unit cost. Foveated sensors (sharp center, coarse periphery, like eyes) make the choice continuous: where to look next is the perception algorithm. The loop it serves is embodied vision; its route-planning cousin is visual navigation (donor table: link when that wave lands).

The one idea you need first. Next-best-view (NBV) scores each candidate camera move by expected information gain minus motion cost: $(\text{entropy before} - \text{expected entropy after}) / \text{cost}$ . Entropy in bits measures the current confusion; a move that collapses $50/50$ odds to $90/10$ earns about half a bit. The policy takes the argmax and re-plans after every observation.

Think of It Like This

Reading a tiny label

Nobody reads a medicine label from across the room by squinting harder. You walk over, tilt it to the light, and lean in — three perception actions chosen to kill specific uncertainties (distance, glare, angle). If the first tilt glares, you tilt the other way; the plan updates per observation.

Active vision is that walk formalized. Zoom is leaning in; exposure bracketing is tilting to the light; circling is checking the back for the ingredient list. Squinting harder is the passive baseline: more parameters staring at the same inadequate pixels.

How It Actually Works

The system maintains a belief (object identity odds, occupancy grid, or pose distribution) and a sensor model predicting what each candidate view would show. For each candidate it computes expected posterior entropy by simulating observations, subtracts motion and time cost, and executes the winner. Foveated designs add a continuous gaze policy: saccade to the highest-gain region, process the fovea finely and the periphery coarsely, repeat until confidence crosses threshold.

Greedy one-step gain works when views are independent; cluttered scenes need short-horizon lookahead, because the best immediate view can strand the camera behind obstacles.

Worked example

Object identity is $50/50$ (entropy $1.0$ bit). Two candidate moves:

Zoom (cost $2$ ): expected odds $90/10$ . Entropy $= -(0.9\log_2 0.9 + 0.1\log_2 0.1) = -(-0.137 - 0.332) = 0.469$ bits. Gain $= 1.0 - 0.469 = 0.531$ bits; rate $= 0.531 / 2 \approx 0.266$ bits per cost.
Pan (cost $1$ ): expected odds $70/30$ . Entropy $= -(0.7\log_2 0.7 + 0.3\log_2 0.3) = -(-0.360 - 0.521) = 0.881$ bits. Gain $= 0.119$ bits; rate $= 0.119$ per cost.

Zoom wins per cost ( $0.266$ versus $0.119$ ) despite costing double — information per freight, not per trip. Raise zoom's cost to $5$ and pan wins instead.

Code

The point of this snippet is the NBV rule: entropy arithmetic picks the view, and cost changes the winner.

1import math2
3
4def entropy(p: float) -> float:5    return -(p * math.log2(p) + (1.0 - p) * math.log2(1.0 - p))6
7
8def rate(before: float, after_p: float, cost: float) -> float:9    return (entropy(before) - entropy(after_p)) / cost10
11
12print(round(rate(0.5, 0.9, 2.0), 3))  # -> 0.26613print(round(rate(0.5, 0.7, 1.0), 3))  # -> 0.119

Watch Out For

Greedy views that strand the camera

One-step gain maximization drives cameras into corners: the close-up behind the door, the zoom with no retreat path. Each move looked optimal alone; the sequence wastes the budget backtracking.

Add short-horizon lookahead (two to three moves) or a coverage bonus for keeping options open. Perception actions consume the same battery as task actions, so plan them jointly, not greedily.

Open-loop plans versus shaky actuators

NBV plans assume the camera lands where commanded. Real pan-tilt heads overshoot, drones drift, and every observation arrives slightly off-pose — silently invalidating the simulated gains the choice rested on.

Close the loop: re-localize after each move and re-plan from the true pose. Budget calibration error into the sensor model rather than assuming commanded equals achieved.

The Quick Version

Active vision chooses camera moves by expected uncertainty reduction per unit cost.
The worked fixture prices zoom at $0.266$ bits per cost against $0.119$ for pan, so zoom wins.
Foveated designs make gaze a continuous policy: sharp center, coarse periphery, saccade to gain.
Greedy one-step choices strand cameras; add lookahead and keep retreat paths open.
Actuators miss commanded poses, so re-localize and re-plan after every move.