Adversarial Examples in Vision

Why Does This Exist?

An image classifier can score above 90 percent on its test set and still call a panda a gibbon with 99 percent confidence after an attacker shifts every pixel by an amount no human can see. That is not a broken model. It is the normal consequence of deciding by boundaries in a high-dimensional space: most photos sit closer to a boundary than intuition suggests, and a nudge aligned with the model's gradients crosses it.

This matters wherever a camera feeds a decision: driver assistance, face gates, content filters. An attacker with the model, or a decent copy of it, manufactures the nudge with a single backward pass. No camera hacking, no stickers required.

This page covers the attack: what the perturbation is, how one gradient step finds it, and why it transfers between models. Training a model to survive such noise is a separate topic in adversarial training, and the confidence scores under attack are the classifier outputs from image classification.

The one idea you need first. A classifier turns pixel values into one score per class and reports the highest. The gradient of the loss with respect to the pixels says which direction in pixel space raises the loss fastest. Training follows that signal to change the weights; an attacker follows it to change the photo while the weights stay frozen.

Think of It Like This

How It Actually Works

Write the photo as a vector $x$ and its true label as $y$. The trained weights $\theta$ stay frozen: nothing is retrained. The attacker wants a perturbation $\eta$ such that the model misreads $x + \eta$, under the cap that no pixel moves more than $\epsilon$. In symbols, $\lVert \eta \rVert_\infty \le \epsilon$: every coordinate shifts by at most $\epsilon$, which is what keeps the change invisible.

The Fast Gradient Sign Method finds $\eta$ in one step. Take the gradient of the loss with respect to the photo itself, $\nabla_x J(\theta, x, y)$, keep only each coordinate's sign, and scale by $\epsilon$:

In plain words: every pixel moves the full allowed amount in whichever direction raises the loss. The sign matters more than the size, because across hundreds of thousands of pixels the tiny pushes accumulate along the model's most sensitive direction.

Worked example: two pixels flip a verdict

Take a toy classifier with two pixel values $x = [1.0, 1.0]$. The true class scores with weights $w_0 = [0.5, 0.0]$ and the rival with $w_1 = [0.2, 0.2]$. Before the attack:

The true class leads 0.50 to 0.40. The margin (rival minus true) grows fastest along $w_1 - w_0 = [-0.3, 0.2]$, whose signs are $[-1, +1]$. One FGSM step with $\epsilon = 0.25$:

New scores: $s_0 = 0.5 \times 0.75 = 0.375$ and $s_1 = 0.2 \times 0.75 + 0.2 \times 1.25 = 0.15 + 0.25 = 0.40$. The rival now leads 0.40 to 0.375, so the label flips, yet each pixel moved only 0.25. A real photo runs the same arithmetic over hundreds of thousands of pixels, which is why $\epsilon$ can stay invisible while the verdict changes completely.

Untargeted attacks accept any wrong answer; targeted ones maximize one chosen wrong class, the gibbon in the famous example. Perturbations also transfer: noise brewed on one model often fools another trained on similar data, so attackers do not always need your exact weights. If you want to know which pixels the model leaned on, the same gradients power the saliency maps in explainability.

Code

The point of this snippet is the flip, not the formula: one capped step along the margin gradient turns a correct verdict into a wrong one on numbers you can check by hand.

1import numpy as np2
3w_true: np.ndarray = np.array([0.5, 0.0])  # weights voting for the right answer4w_rival: np.ndarray = np.array([0.2, 0.2])  # weights voting for the wrong answer5x: np.ndarray = np.array([1.0, 1.0])6
7print(w_true @ x, w_rival @ x)  # -> 0.5 0.48
9step: np.ndarray = 0.25 * np.sign(w_rival - w_true)  # one FGSM step, capped at 0.2510x_adv: np.ndarray = x + step11print(x_adv)  # -> [0.75 1.25]12print(w_true @ x_adv, w_rival @ x_adv)  # -> 0.375 0.4

The true score falls from 0.50 to 0.375 while the rival holds 0.40, so argmax switches sides. Every input moved by at most 0.25, the $\epsilon$ budget from the worked example.

Watch Out For

The Quick Version

• An adversarial example is a normal input plus a tiny crafted perturbation that flips the model's answer.
• FGSM finds the perturbation in one step: each input moves by $\epsilon$ along the sign of the loss gradient.
• The worked toy flips a verdict from 0.50 versus 0.40 to 0.375 versus 0.40 with every pixel moving at most 0.25.
• Targeted attacks aim at one chosen wrong class; untargeted attacks accept any wrong answer.
• Perturbations transfer between models trained on similar data, so hiding the weights is not a defense.
• Clean accuracy says nothing about robustness; evaluate at an explicit epsilon budget and train on perturbed inputs.

What to Read Next

• Image Classification
• Uncertainty Estimation in Vision
• Vision Explainability Overview