Colorization

Colorization predicts believable colours for grey photos by learning which hues typically dress each object, sky blue, grass green, skin warm.

Why Does This Exist?

Archives hold billions of grey photographs whose colours are lost to chemistry and time. Manual colorization takes artists hours per image and injects modern guesses as false history. An automatic first pass that dresses skies blue and uniforms plausibly khaki turns weeks of work into an afternoon of review.

The task is fundamentally ambiguous: many colourings fit one grey image. Regression to the mean produces tasteful desaturation, classification over colour bins with rebalancing produces vivid plausible hues. The task map is image-to-image tasks.

Think of It Like This

A costume designer for a black-and-white film

A costume designer receives a grey film still and must dress the actors. Soldiers get khaki, skies stay out of wardrobe's hands, but the lead's car could be red or midnight blue with equal evidence. The designer picks the choice audiences accept, knowing history might disagree.

The network is the designer and the training set is its memory of past films. It fails on period costumes it never saw, dressing Romans in denim, which is why archival review stays mandatory.

How It Actually Works

Work in Lab space: input lightness $L$ , predict colour channels $a, b$ . Classification approaches quantise $ab$ into $313$ bins and predict a distribution per pixel, trained with class-rebalanced cross-entropy that upweights rare vivid colours against dominant greys and browns:

p(a, b \mid L), \qquad L_{\text{rebalanced}} = -\sum w(a,b) \log p(a,b)

At inference, annealed-mean sampling blends distribution sharpness: full mean gives desaturated safety, mode gives vivid risk. Regression variants predict $ab$ directly and desaturate; GAN variants add a realism critic for punch.

Worked example

Grey car, $L = 60$ . The network's distribution says red bins $0.35$ , blue $0.30$ , grey $0.20$ , other $0.15$ . Annealed mean picks a muted red-brown: plausible, possibly wrong, always colourful. A regression model outputs the expectation, a grey mauve no car ever wore. Rebalancing weights make red count triple its frequency, without which every car comes out grey.

Code

Annealed choice between safe average and vivid mode.

1import numpy as np2
3
4def annealed_pick(probs: np.ndarray, temp: float = 0.38) -> int:5    """Sample a colour bin from a temperature-sharpened distribution."""6    sharp = probs ** (1 / temp)  # low temp concentrates mass7    sharp = sharp / sharp.sum()8    return int(np.argmax(sharp))  # mode of the sharpened vote9
10
11p = np.array([0.35, 0.30, 0.20, 0.15])12print(annealed_pick(p))  # -> 0 the leading red bin wins

Watch Out For

Presenting plausible as historical

Colorized archives read as truth to viewers: that uniform was khaki, that car was red. The model guessed, and confident guesses fossilise into false memory, especially skin tones and insignia where errors carry weight.

Watermark colorized outputs as interpretive and keep originals attached. Review by domain experts, not just by aesthetics.

Desaturation from unweighted regression

Plain L2 on $ab$ channels averages the rainbow into tasteful mud. Every output looks developed in the same grey-brown lab because the loss rewards hedging.

Use rebalanced classification or perceptual losses for vividness. If everything arrives sepia, the loss, not the data, is the suspect.

The Quick Version

Colorization predicts $ab$ colour from $L$ brightness in Lab space.
Rebalanced classification over $313$ bins beats regression's desaturated averages.
Our grey car splits red $0.35$ , blue $0.30$ , and annealed sampling picks vivid red.
Ambiguity is fundamental: many colourings fit one grey input.
Label outputs interpretive, and fix sepia-everything with rebalanced losses.