Physically Grounded Generation

Generators paint seven-fingered hands and floating chairs: pretty, impossible. Physically grounded generation constrains pixels with geometry, contact, and gravity.

A 15 centimeter air gap fails the 2 centimeter support tolerance while contact passes, filtering floating furniture automatically

Why Does This Exist?

Unconstrained generators ship impossibilities: chairs hovering inches up, hands merging with cups, staircases to nowhere. Pretty pixels, broken worlds — useless for simulation data, AR placement, and robotics, where downstream systems trust geometry the generator never checked. Photorealism without physics is a facade program.

Physically grounded generation conditions synthesis on structure (depth, normals, layouts) and filters outputs through physics (support, contact, gravity, articulation limits). The reasoning conscience is physical reasoning; the spacetime sibling is 4D understanding.

The one idea you need first. A support check verifies every object rests on something: trace down from each object's lowest point; contact within tolerance (millimeters for tabletops, centimeters for floors) passes, air gaps fail. Contact reasoning extends it to hands (grasp closures), stacking (load paths), and articulation (joint limits). Checks run on generated depth, not pixels — geometry judges geometry.

Think of It Like This

A theater set versus a building

A theater set shows a staircase ascending into canvas: convincing from row H, plywood and paint from backstage. A building's staircase stands from every angle because structure, not viewpoint, holds it up. Audiences forgive sets; tenants would not live in one.

Ungrounded generators build sets: one-camera facades that collapse under viewpoint change or physical use. Grounded generation submits plans to the building inspector (support, contact, gravity) before painting. Fewer opening nights, zero collapses.

How It Actually Works

Conditioning comes first: diffusion and autoregressive models take depth maps, normal maps, bounding layouts, or 3-D boxes alongside text, so structure guides denoising from step one. Filtering comes second: generated scenes pass support/contact simulators that reject or repair violations (snap to contact, resample backgrounds). Losses close the loop: contact and penetration penalties during training teach generators the inspector's taste. Evaluation renders multiple viewpoints and runs physics checks, since single-view metrics bless facades.

The strictness dial trades diversity for validity; production sets it per use (sim data strict, art loose).

Worked example

A generated chair: seat bottom at $60$ cm, floor at $45$ cm (seat height above a $45$ -cm platform edge — read as the supporting surface). Air gap: $60 - 45 = 15$ cm. Tolerance: $2$ cm. Since $15 > 2$ , the chair floats: reject or snap down $15$ cm.

A second chair: bottom at $45.5$ cm over the same $45$ -cm surface. Gap $0.5$ cm $\le 2$ cm: contact passes. The filter is one subtraction per object — trivial arithmetic that catches what billion-parameter models miss, because the models never learned to subtract.

Code

The point of this snippet is the inspector: support gaps in, verdicts out.

1def support_check(bottom_cm: float, surface_cm: float, tol_cm: float = 2.0) -> str:2    gap = bottom_cm - surface_cm3    return "contact" if gap <= tol_cm else "floating"  # inspector's verdict4
5
6print(support_check(60.0, 45.0))  # -> floating7print(support_check(45.5, 45.0))  # -> contact

Watch Out For

Strictness that sterilizes output

Hard physics filters reject the unusual along with the impossible: Cantilever chairs, action poses, and artistic compositions all trip naive support checks. Over-constrained generators produce sterile catalog renders — valid, boring, and blind to legitimate edge cases.

Calibrate tolerance per category and allowlist known-valid exceptions (cantilevers, jumps, held objects). The inspector should catch impossibilities, not enforce mediocrity.

Proxies that disagree with perception

Depth estimators and normal predictors err systematically (reflective surfaces, thin structures), so physics checks on estimated geometry reject valid images and pass broken ones. The inspector's instruments need their own calibration.

Validate the checker chain on real photos first: it must pass nearly everything real before judging synthetic. A filter failing reality filters fantasy randomly.

The Quick Version

Physically grounded generation conditions on structure and filters through support and contact checks.
The worked fixture rejects a $15$ -cm hover and passes a $0.5$ -cm contact against a $2$ -cm tolerance.
Depth, normals, and layouts guide denoising from step one; simulation verdicts close the loop.
Calibrate strictness per use: sim data strict, art loose, exceptions allowlisted.
Validate checkers on real photos first or proxies condemn valid scenes.