CutMix Augmentation

Paste a square from photo B into photo A and mix the labels by area. The model sees real pixels from both scenes with an honest mixed tag.

Pasting a 16 by 16 patch into a 32 by 32 image keeps 75 percent of the first photo, so the label becomes 0.75 and 0.25.

Why Does This Exist?

Mixup blends every pixel, so no region stays real. Localization suffers because the model never sees a clean edge from either scene, only ghosts. Detection and classification both want mixed supervision without giving up real pixels.

CutMix keeps both. It cuts a rectangle from image $B$ , pastes it into image $A$ , and sets the label from the kept area. The result shows true $A$ pixels outside the patch and true $B$ pixels inside, with a label that says exactly how much of each is visible. The family overview is image augmentation, and the erase only cousin is cutout.

You need one idea first. Area is the label. The pasted fraction sets the mixed tag, not a sampled weight.

Think of It Like This

A collage with the receipt attached

You glue a magazine square onto a newspaper page and attach a receipt saying 75 percent newspaper and 25 percent magazine. Nobody claims the collage is one pure source.

CutMix is that collage. The binary mask is the glue edge, and the area ratio is the receipt. Where the analogy stops: the model must learn from the collage as if both sources stay present, not as a new single scene.

How It Actually Works

Mask and label rule

Sample a binary mask $\mathbf{M}$ with a random rectangle set to $0$ inside and $1$ outside. Then:

\tilde{x} = \mathbf{M} \odot x_A + (1 - \mathbf{M}) \odot x_B, \quad \tilde{y} = \lambda y_A + (1 - \lambda) y_B

Here $\lambda$ is the kept area fraction of $A$ , so $\lambda = 1 - (\text{patch area} / \text{image area})$ . Unlike mixup, $\lambda$ comes from geometry, not from a Beta draw.

A worked area label

Image $32 \times 32 = 1024$ . Patch $16 \times 16 = 256$ :

\lambda = 1 - 256 / 1024 = 0.75

Label $[1, 0]$ and $[0, 1]$ become $[0.75, 0.25]$ . Pixels outside the patch are pure $A$ , pixels inside are pure $B$ , and the tag matches the visible split.

Patch sampling

Patch width and height come from sampling $\lambda$ uniformly, then sizing the box to that area with matching aspect jitter. Clip the patch to the frame. For detection, boxes under the patch need reweighting or dropping when mostly covered.

Code

1import numpy as np2
3
4def cutmix_label(patch: int, image: int) -> float:5    """Kept fraction of A after pasting a square patch. Returns lambda."""6    return 1.0 - (patch * patch) / float(image * image)7
8
9print(cutmix_label(16, 32))  # -> 0.75

Watch Out For

Pasting over the only small object

A $16 \times 16$ patch can fully bury a tiny bird while the label still claims $0.75$ bird. The model learns background as bird. The symptom is falling small object recall after CutMix helps large ones. Shrink patches on small object datasets or drop covered instances.

Treating CutMix like dropout at test time

CutMix collages never appear at deployment, so heavy use shifts the train distribution. The symptom is great train mixing accuracy with flat validation. Keep CutMix probability near $0.5$ and always validate on clean images.

The Quick Version

CutMix pastes a real patch from one photo into another and labels by kept area.
A 16 by 16 patch in a 32 by 32 image gives label 0.75 and 0.25.
Lambda comes from patch geometry, not from a random Beta draw.
Shrink patches or drop covered boxes when small objects can vanish underneath.