GAN Inversion

GAN inversion runs a generator backwards, searching for the latent code that rebuilds your photo so you can edit it with the model's own controls.

Inversion searches latent space until the generator rebuilds the input photo, then edits move along discovered attribute directions

Why Does This Exist?

Generators dream up new faces brilliantly but cannot touch your photo. Their controls, age sliders, pose knobs, style codes, live in latent space, and real photos arrive with no latent code attached. The most powerful editing machinery in vision sat locked behind that missing address.

Inversion finds the address. Optimise a latent vector until the generator's output matches the photo, then every latent control works on it: age the face, rotate the head, relight the scene, all with the generator's learned realism. The generator most often inverted this way is StyleGAN, whose layered codes make the search behave.

Think of It Like This

Finding the recipe for a finished dish

You taste a superb soup and want a less salty version. The kitchen only accepts recipe cards, not bowls of soup. So you work backwards: guess ingredients, cook, taste, adjust, until your pot matches theirs spoon for spoon. Now you hold their recipe card, cross out half the salt, and cook the fixed soup.

The photo is the finished soup and the latent code is the recipe card. Optimisation is the guess-and-taste loop. It fails on dishes the kitchen never learned, out-of-domain photos no recipe can rebuild, which is the domain limit below.

How It Actually Works

Given photo $x$ and frozen generator $G$ , solve:

w^* = \arg\min_w \; \lVert G(w) - x \rVert_2^2 + \lambda_{\text{p}} \, \mathrm{LPIPS}(G(w), x)

LPIPS is perceptual distance through fixed classifier features, which forgives pixel shifts humans ignore. Optimisation runs hundreds of gradient steps from an encoder's quick guess, in the extended $W^+$ space where each layer gets its own code for maximum fidelity. Editing then moves $w^*$ along precomputed attribute directions: $w_{\text{older}} = w^* + 2.0 \cdot d_{\text{age}}$ .

Worked example

A portrait starts at LPIPS $0.42$ from a random code. After $500$ optimisation steps the reconstruction reaches LPIPS $0.08$ , visually near-identical. Moving $+2.0$ along the age direction adds apparent years while identity holds; pushing to $+6.0$ breaks identity as the code leaves the well-modelled region. The usable edit range, about $\pm 3$ units, is the practical measure of inversion quality, not the reconstruction score alone.

Code

Inversion loop skeleton, showing encoder warm-start plus refinement.

1def invert(steps: int = 500, start_lpips: float = 0.42) -> float:2    """Toy decay of perceptual error across refinement steps."""3    return round(start_lpips * (0.99 ** (steps - 1)), 2)4
5
6print(invert())  # -> 0.0 toy floor; real runs plateau near 0.08

Watch Out For

Perfect pixels with an uneditable code

Optimisation in an over-flexible space can match pixels with a code far from anything the generator understands. Reconstruction scores dazzle while every edit direction produces garbage, because the code sits off the semantic manifold.

Prefer encoder-regularised codes in $W^+$ over raw pixel chasing, and test editability, not just reconstruction. A useful inversion edits cleanly at $\pm 3$ units.

Inverting out-of-domain photos

Feed a cartoon, a profile view, or heavy occlusion to a face generator and optimisation strains toward the nearest face-like output. The result resembles the input the way a caricature resembles its subject: recognisable, wrong, and uneditable.

Check domain first: frontal, clear, face-like inputs invert well; everything else needs a generator trained on that domain.

The Quick Version

GAN inversion recovers the latent code whose generation matches a real photo.
Objective mixes pixel MSE with perceptual LPIPS distance over hundreds of steps.
Our portrait drops from LPIPS $0.42$ to $0.08$ across $500$ refinement steps.
Edits move the code along attribute directions within about $\pm 3$ units.
Pixel-perfect but uneditable codes and out-of-domain inputs are the two traps.