Image Outpainting

Image outpainting extends a photo beyond its borders, inventing plausible surroundings that continue the scene's perspective, light, and texture.

Outpainting invents new columns on both sides that continue the photo's perspective and lighting

Why Does This Exist?

Frames crop reality. A portrait shot vertical cannot fill a widescreen banner, and reshooting is often impossible: the moment passed, the set is gone. Cropping tighter destroys composition; stretching distorts faces.

Outpainting invents the missing surroundings conditioned on the visible photo. Unlike inpainting, which fills holes surrounded by context on all sides, extrapolation works from one edge with no far-side anchor, a strictly harder prediction. The mask-based sibling is owned by image editing and inpainting, and the task map is image-to-image tasks.

Think of It Like This

A muralist finishing a torn fresco edge

A fresco survives only in its centre; both edges crumbled away. The restorer extends the sky leftward and the meadow rightward, matching the surviving brushwork stroke for stroke at the seams, then improvising further out where no evidence constrains anything.

Near the seam, fidelity rules: continue lines, light, and texture exactly. Far from it, plausibility rules: any fitting sky will do. The work fails at the seam first, mismatched horizons betraying the join, which is why seam metrics matter more than far-field beauty.

How It Actually Works

Mask the known centre and generate outward, either in one pass with a wide-context model or iteratively in strips. Conditioning carries perspective lines, illumination direction, and texture statistics across the boundary; diffusion methods denoise the unknown region while clamping known pixels each step, the same clamping trick as inpainting but anchored on one side only:

\text{known} = 512 \text{ px}, \qquad \text{new} = 2 \times 128 \text{ columns} = 256 \text{ px}, \qquad \text{total} = 768 \text{ px}

Seam quality decides success: gradient continuity across the boundary, matched noise grain, consistent vanishing points. Iterative strips accumulate drift, so single-pass wide models win where they fit in memory.

Worked example

Extend $512$ to $768$ wide: $128$ invented columns per side, a third of the final image unseen. Near-seam strips repeat the photo's column statistics within $5$ percent on brightness and contrast; far strips diverge freely. A horizon line at row $300$ must continue at row $300 \pm 2$ across the seam, a tolerance that exposes weak models instantly while viewers forgive invented clouds further out.

Code

Extension geometry that sizes the invented region.

1def outpaint_geometry(known: int = 512, per_side: int = 128) -> tuple[int, float]:2    """Total width and invented fraction for symmetric outpainting."""3    total = known + 2 * per_side4    return total, round(2 * per_side / total, 2)5
6
7print(outpaint_geometry())  # -> (768, 0.33) a third of the image invented

Watch Out For

Seams that betray the join

Horizon jumps, grain changes, and light-direction flips at the boundary scream synthesis. Viewers forgive invented content but never visible seams, and metrics averaged over the whole image hide boundary failures.

Score seam bands separately from far fields. Feathered blending and gradient-matching losses target the join directly.

Perspective the model cannot continue

Wide-angle lines converge toward vanishing points outside the frame. Models without geometric understanding extend lines straight past where they should bend, producing subtly impossible architecture.

Check continuations of strong lines first. For architecture-heavy scenes, condition on estimated perspective or vanishing points explicitly.

The Quick Version

Outpainting extrapolates beyond borders with context anchored on one side only.
Our extension grows $512$ to $768$ wide with $256$ invented columns, a third of the image.
Seam continuity matters more than far-field beauty; score the join separately.
Diffusion clamping works like inpainting but with half the anchoring evidence.
Perspective errors and visible seams are the two characteristic failures.