Super-Resolution and Image Restoration

Super-resolution rebuilds a sharp high-resolution image from a blurry or noisy one, learning what plausible detail looks like instead of just smoothing pixels.

A restoration network inverts blur and downsampling, growing a 64 pixel input into a 128 pixel output with learned texture.

Why Does This Exist?

Cameras blur, sensors add noise, and compression throws away texture. Zooming a small crop, recovering an old photograph, or reading a compressed scan all ask the same thing: reconstruct sharp detail from a degraded input. Plain interpolation only smears the remaining pixels across a bigger canvas, so edges stay soft and texture never returns.

Deep restoration learns what sharp images look like and paints in plausible high-frequency detail: pores, foliage, brickwork. That power splits by task. Image denoising cleans noise at the same size, image enhancement adjusts tone and contrast, and this page grows resolution and reverses degradation together.

This page covers the degradation model, a hand-computed 2 by 2 example, and the hallucination trade that bounds the whole family.

Think of It Like This

A conservator filling a damaged mosaic

A conservator faces a mosaic with missing tiles. They do not smear the surviving colors outward. They study the intact border pattern, recall how motifs from that era complete themselves, and cut new tiles that continue the design tile by tile.

The network is that conservator: the surviving low-resolution pixels are the intact border, the training set is the memory of motifs, and each synthesized texture patch is a cut tile. The analogy stops at knowledge. A conservator knows the building's era and workshop. The network only knows its training distribution, so out-of-distribution inputs get tiles from the wrong building.

How It Actually Works

The degradation model

Restoration treats imaging as a forward corruption to invert: $y = (x \ast k)\downarrow_s + n$ . Here $x$ is the sharp high-resolution image, $k$ is the blur kernel, $\downarrow_s$ is downsampling by factor $s$ , $n$ is sensor noise, and $y$ is the degraded observation you actually receive. Training pairs are built by corrupting sharp images this way, then the network learns the reverse mapping from $y$ back toward $x$ . The body is a convolutional or transformer feature extractor plus an upsampler (pixel shuffle or transposed convolution), and the loss mixes three terms: pixel L1 for fidelity, perceptual loss on deep feature maps for natural textures, and an adversarial term that pushes outputs toward the manifold of real photos.

Worked 2 by 2 numbers

Take a sharp 2 by 2 patch $\begin{bmatrix} 10 & 20 \\ 30 & 40 \end{bmatrix}$ . A 2 by 2 box blur followed by 2x downsampling collapses it to one pixel: $(10 + 20 + 30 + 40) / 4 = 25$ . Suppose the network predicts $\begin{bmatrix} 12 & 18 \\ 28 & 42 \end{bmatrix}$ . The mean absolute error is $(2 + 2 + 2 + 2) / 4 = 2.0$ , honest fidelity, while a blur-only baseline predicting all $25$ s scores $(15 + 5 + 5 + 15) / 4 = 10.0$ . Pixel shuffle performs the growth step exactly: four 1 by 1 channels holding $[10, 20, 30, 40]$ at one location rearrange into that 2 by 2 block, no interpolation blur involved.

Why PSNR alone misleads

Peak signal-to-noise ratio rewards pixel-exact averages, so the blurry all- $25$ s prediction can outscore a sharp prediction whose texture sits one pixel off. Optimizing PSNR alone therefore breeds soft, safe outputs. The adversarial and perceptual terms buy crispness at the cost of occasional invented texture: the perception-distortion trade, paid in hallucination risk wherever ground truth is unknowable.

Code

1def nearest_upsample(grid: list, factor: int) -> list:2    """Grow a grid by repeating each pixel into a factor by factor block."""3    out = []4    for row in grid:5        wide = [v for v in row for _ in range(factor)]6        for _ in range(factor):7            out.append(list(wide))8    return out9
10
11def mean_abs_error(a: list, b: list) -> float:12    """Mean absolute error between two equal-size grids."""13    flat_a = [v for row in a for v in row]14    flat_b = [v for row in b for v in row]15    return round(sum(abs(x - y) for x, y in zip(flat_a, flat_b)) / len(flat_a), 2)16
17
18sharp = [[10, 20], [30, 40]]19pred = [[12, 18], [28, 42]]20print(nearest_upsample([[25]], 2))21# -> [[25, 25], [25, 25]]22print(mean_abs_error(pred, sharp))23# -> 2.024print(mean_abs_error([[25, 25], [25, 25]], sharp))25# -> 10.0

Watch Out For

Invented detail presented as evidence

Super-resolved outputs look like photographs, so teams file them as facts: a sharpened plate number in a report, an enhanced lesion in a diagnosis. The model painted the most plausible texture, not the true one.

Treat every output as a plausible reconstruction, never ground truth, in medical, forensic, and security work. Keep the original alongside the restoration and flag regions the model filled in.

Chasing PSNR into blur

A model tuned on PSNR alone wins the leaderboard with soft outputs: blur minimizes pixel error while looking nothing like a sharp photo. The symptom is rising scores beside visibly mushy textures.

Add perceptual and adversarial terms and judge with LPIPS or human raters next to PSNR. Ship the model people prefer, not the one the single metric prefers.

The Quick Version

Restoration inverts $y = (x \ast k)\downarrow_s + n$ : blur, downsample, and noise, learned backwards.
Pixel shuffle grows resolution by rearranging channels, with no interpolation smear.
Worked patch: 2 by 2 averaging collapses to 25, and a prediction with error 2.0 beats blur at 10.0.
Pixel loss buys fidelity while perceptual and adversarial losses buy crispness.
PSNR rewards safe blur, so judge sharpness separately and never file invented texture as fact.