Image-to-Image Tasks

Image-to-image tasks transform a picture into another picture, denoised, sharpened, recolored, or restyled, while keeping its content recognisable.

Why Does This Exist?

Newcomers face a wall of named methods with no map. Denoising, deblurring, super-resolution, inpainting, colorization, style transfer: each has its own datasets, metrics, and failure modes, yet all share one contract, pixels in, pixels out, content preserved. Without the map, teams apply GAN translators to denoising or regression to style transfer and wonder at the wreckage.

This page is the map. Three families organise the territory by what changes: restoration recovers a true original, enhancement improves without a ground truth, stylization deliberately departs from reality. The paired-data workhorse is Pix2Pix, the unpaired answer is CycleGAN, and the GAN family umbrella is image to image GANs.

Think of It Like This

A photo lab with three counters

A photo lab serves three counters. Restoration removes scratches from grandma's photo, aiming at the true original. Enhancement brightens your dull vacation shots, aiming at pleasing with no original to match. Stylization turns your portrait into a watercolor, aiming away from reality on purpose.

One machine cannot serve all counters: the scratch remover needs truth, the brightener needs taste, the watercolorist needs nerve. Method choice starts by walking to the right counter, and every pitfall below is a story of the wrong queue.

How It Actually Works

Every task maps input $x$ to output $y$ with a content-preservation constraint measured against task truth. Restoration tasks (image denoising, image deblurring, image dehazing, super-resolution) score against known originals with PSNR and SSIM. Enhancement tasks (low-light enhancement, HDR imaging) lack ground truth and use no-reference quality or human preference. Stylization tasks (style transfer, colorization) trade fidelity for effect and need perceptual or task-specific metrics.

Data regime picks the method family. Paired aligned data, $400$ Facades sketches with photos, suits direct regression plus adversarial sharpening. Unpaired collections, $1{,}000$ horses plus $1{,}000$ zebras, need cycle consistency or shared-latent tricks. Outpainting extends borders (image outpainting) and instruction edits rewrite regions (image editing).

Worked example

Two dataset cards decide two methods. Facades: $400$ aligned pairs, L1-valid, Pix2Pix territory, expected sharp outputs following lines. Horse-zebra: $1{,}000$ plus $1{,}000$ unpaired, cycle territory, expected texture transfer with layout preserved. Swap the methods and both fail: Pix2Pix on unpaired data has no valid L1 target, CycleGAN on paired data wastes its round-trip machinery and underperforms direct supervision.

Watch Out For

Scoring stylization with restoration metrics

PSNR between a watercolor and its photo is meaningless, yet pipelines default to it. Optimising the wrong metric then punishes the best stylizations for departing from pixels they were meant to leave.

Pair each family with its metric: PSNR/SSIM for restoration, no-reference or human ratings for enhancement, perceptual and preference scores for stylization.

Assuming content preservation is automatic

Nothing in adversarial training preserves content by default; only explicit terms, L1, cycle, identity, do. Remove them chasing sharper outputs and translations drift into plausible unrelated images.

Keep one content anchor in every objective and ablate it deliberately. Sharp garbage is still garbage.

The Quick Version

Image-to-image maps pictures to pictures under a content-preservation contract.
Restoration recovers truth, enhancement pleases without truth, stylization departs deliberately.
Paired aligned data suits Pix2Pix; unpaired collections need cycle consistency.
Facades' $400$ pairs versus horse-zebra's $1{,}000$ plus $1{,}000$ unpaired pick the method.
Match metrics to families, and never drop the content anchor chasing sharpness.