Computational Photography

Phone cameras beat physics with math: tiny lenses, noisy sensors, yet stunning night shots. Computational photography stacks bursts, merges exposures, and learns priors to outshoot glass.

Eight-frame bursts cut noise nearly threefold while weighted HDR merging lands at 96 from three exposures

Why Does This Exist?

Phone bodies cannot carry glass: millimeter-thick lenses and pea-sized sensors cap light collection a hundredfold below full-frame cameras. Physics says phone photos should be noisy, blurry, and range-starved — and single shots are. Yet night modes stun, because the shutter fires bursts while algorithms do the seeing.

Computational photography replaces optics with computation: align-and-merge bursts for noise, multi-exposure fusion for range, subpixel dithering for resolution, learned denoisers and tone maps for finish. The shot is a computation with a lens attached. The ray-recording sibling is light field imaging; the measurement-frugal sibling is compressive sensing.

The one idea you need first. Align-and-merge registers a burst to one reference frame, then averages: signal adds coherently while independent noise averages down by $\sqrt{N}$ for $N$ frames. HDR merging weights each exposure's pixels by well-exposedness (downweighting clipped highs and noisy lows) so every tone comes from its best witness.

Think of It Like This

A choir versus a soloist

One great soprano outsings any amateur — a single full-frame exposure beats any single phone frame. But eight decent singers in tight harmony produce a wall of sound no solo throat matches: voices add, wobbles cancel, and the blend covers individual cracks.

Burst photography is the choir. Each frame is a mediocre singer (noisy, dim); alignment is the rehearsal putting everyone on the same note; averaging is the performance where signal harmonizes and noise — singing random notes — cancels itself. HDR adds sections: basses (shadows) and sopranos (highlights) each sing their best range.

How It Actually Works

Capture fires a burst around the shutter press (including pre-shutter frames from the always-running viewfinder stream). Pipeline stages: pick the sharpest reference, align the rest hierarchically (global motion, then tile-level flow), merge with robustness weights that reject misaligned tiles, then demosaic, denoise with learned priors, tone-map, and sharpen. Night modes extend exposures and lean harder on alignment; super-resolution modes exploit subpixel handshake jitter as free dithering.

Motion breaks the harmony assumption: movers need detection and single-frame fallback, or they smear into ghosts.

Worked example

Noise: single-frame noise $\sigma$ ; $8$ -frame merged noise $= \sigma/\sqrt{8} = \sigma/2.83$ — nearly threefold cleaner, the choir dividend.

HDR: one scene point reads $250$ (near clipping), $120$ , and $30$ across three exposures. Well-exposedness weights $0.1$ , $1.0$ , $0.6$ :

$\frac{0.1 \times 250 + 1.0 \times 120 + 0.6 \times 30}{0.1 + 1.0 + 0.6} = \frac{25 + 120 + 18}{1.7} = \frac{163}{1.7} \approx 95.9$

Merged value $\approx 96$ : the mid exposure dominates, the clipped one barely votes, the dark one assists. Every tone elects its best witness.

Code

The point of this snippet is the merge: weights elect exposures and the average lands at $96$ .

1def hdr_merge(values: list, weights: list) -> float:2    total = sum(weights)3    return sum(v * w for v, w in zip(values, weights)) / total4
5
6print(round(hdr_merge([250.0, 120.0, 30.0], [0.1, 1.0, 0.6]), 1))  # -> 95.97print(round(8.0**0.5, 2))  # -> 2.83

Watch Out For

Ghosts where alignment surrendered

Movers defeat tile alignment: pedestrians smear, wheels duplicate, flags shred. Robustness weights downweight the damage, but large motion leaves visible ghosts or fallback seams that read as artifacts in prints.

Detect motion magnitude per tile and fall back to the single best frame there. Review bursts with motion before shipping the merge — the failure hides at thumbnail scale and surfaces at full size.

Watercolor faces from heavy priors

Aggressive learned denoisers erase pores, hair strands, and fabric weave along with noise, leaving waxy "watercolor" skin. The metric (noise down!) improves while the portrait dies.

Tune denoising strength per content (faces weaker than skies) and A/B at $100\%$ crop with human raters. Noise numbers cannot see texture murder; eyes can.

The Quick Version

Computational photography trades glass for bursts: align, merge, and let math outshoot optics.
Eight frames cut noise $2.83$ -fold; the worked HDR merge lands at $96$ from three exposures.
Night modes, super-resolution bursts, and learned finishers extend the same pipeline.
Movers break alignment into ghosts — detect motion and fall back per tile.
Heavy denoisers watercolor faces; judge strength at full crop with human eyes.