Inverse Rendering Decomposes Images

Inverse rendering splits a photo back into shape, material, and light so each can be edited separately.

Why Does This Exist?

Forward rendering turns shape plus material plus light into pixels. Editing needs the reverse: change the bulb without repainting the wall. Image formation teaches the forward path; inverse rendering factors $I = A \cdot S(N, L) + \text{specular}$ into albedo $A$ , shading $S$ from normals $N$ and light $L$ . The factorization is ambiguous without priors.

Think of It Like This

Unmixing paint from lamp light

A beige wall under yellow lamplight photographs orange. Knowing the bulb is yellow lets you divide the photo back to beige paint. Without the bulb note, beige paint under white light looks identical to white paint under yellow light.

Albedo is paint and shading is lamplight. The analogy stops at geometry: real shading also bends with surface angle.

How It Actually Works

Lambertian model writes $I = A \cdot \max(0, N \cdot L) + \epsilon$ per pixel. Unknowns per pixel are $A \in \mathbb{R}^3$ , $N \in S^2$ , plus global $L$ . Priors prefer smooth $N$ , sparse $A$ edges, and low-frequency $L$ in spherical harmonics. Optimization minimizes $\lVert I - \hat{I} \rVert^2$ plus those priors through differentiable rendering.

Worked example

Pixel reads $[0.60, 0.45, 0.30]$ with normal $(0, 0, 1)$ and light $(0.4, 0.3, 0.86)$ normalized. Shading is $N \cdot L = 0.86$ . Albedo recovers as $[0.60, 0.45, 0.30]/0.86 = [0.70, 0.52, 0.35]$ . Relighting with overhead $L' = (0, 0, 1)$ gives $[0.70, 0.52, 0.35]$ , brighter and neutral. Same pixel under the old light would misread as darker paint.

Watch Out For

Baking shadows into albedo texture

A shadowed floorboard optimized without shadow priors darkens $A$ permanently, so relighting keeps a ghost shadow. The symptom is shadows that follow objects after light moves. Add explicit shadow or visibility terms before trusting albedo.

The Quick Version

Inverse rendering factors pixels into albedo, normals, and light.
The split is ambiguous, so smoothness and sparsity priors decide.
Relighting and material edits need the factors, not just pixels.
Do not trust albedo from one photo under harsh shadows alone.
Lambertian $A \cdot (N \cdot L)$ is the starting model before gloss.