Patch Attacks on Vision Models

Concentrate the whole attack into one small sticker the model cannot ignore, placed anywhere in the scene.

A 4 percent sticker optimized once flips 78 of 100 images at any placement, without touching other pixels

Why Does This Exist?

Pixel attacks like FGSM and PGD touch every pixel a little, which no sticker printer can reproduce. Patch attacks invert the budget: leave 96 percent of pixels untouched and spend everything on a small region. The threat model is placement freedom with full training-time optimization, and the payoff is a printable artifact that works across scenes. The bridge to the street is physical-world attacks, and the map is adversarial attacks.

Think of It Like This

One loud heckler in a quiet hall

A debater can be drowned out without rewiring the hall's speakers: one heckler with a megaphone near any seat derails the speech. The patch is the heckler, small in area and total in effect. The analogy stops at locality: ushers can remove hecklers once spotted, while detectors struggle to ignore a patch without also ignoring real objects near it.

How It Actually Works

Optimize patch pixels to force a target prediction across a distribution of backgrounds, scales, rotations, and placements, leaving all other pixels fixed. Training applies the patch at random positions per step so the result works wherever stuck. Larger patches flip more reliably; targeted classes need bigger patches than untargeted flips.

Worked example with recomputed numbers

Patch covering 4 percent of image area, about 45 by 45 on a 224 frame, trained over 5,000 placements. It flips 78 of 100 test images to the target class at random placements, 86 when centered on the main object, 64 at corners. Halve the area to 2 percent and flips fall to 51. Double to 8 percent and flips reach 91 but the sticker is obvious at a glance, pricing stealth against strength in area points.

Defense condition

Defenses must detect or neutralize localized high-confidence regions without deleting real small objects, which is why patch defenses report per-size curves. A defense certified at 2 percent area says nothing about 8 percent.

Watch Out For

Testing patches at one fixed position

A patch tuned and tested dead-center scores 86 and falls to 64 in corners, and deployments meet corners. The symptom is lab strength that vanishes on a dashboard camera. Train and evaluate over random placements, rotations, and lighting, and report the corner number beside the center one.

The Quick Version

All perturbation concentrated in one small printable region, rest untouched.
The 4 percent patch flips 78 anywhere, 86 centered, 64 at corners.
Area prices the trade: 2 percent flips 51, 8 percent flips 91 but visible.
Train over placement distributions or the patch overfits one position.
Defenses answer per patch size, since size decides the game.