Skip to content
AI360XpertBeta

Adversarial Training

Train on your worst-case inputs, not just the clean ones, so a tiny hostile nudge can't flip your model's answer.

Adversarial training crafts the worst small perturbation that flips a prediction, then trains on it with the correct label
Adversarial training crafts the worst small perturbation that flips a prediction, then trains on it with the correct label

Why Does This Exist?

A classifier can score high on clean test images yet flip its answer when you shift a few pixels by an amount you can't see. That flip is not bad luck. The boundary sits close to the data in some direction, and an attacker searches exactly that direction. You'll meet the attack itself in adversarial examples.

The obvious patch doesn't work. Adding random jitter during training almost never steps in the attacker's chosen direction, so the weak spot survives. You have to generate the hostile version of each batch while you train and teach the model the true label for both copies.

This page teaches that defense: the min-max setup, one FGSM step worked by hand, and why practice settled on PGD. It assumes you know which way a loss climbs. If that feels shaky, read gradients first, then come back.

Think of It Like This

A sparring partner who aims at your dropped guard

You box well against a heavy bag. It never hits back, so your dropped left hand never costs you. Then a real opponent notices it in the first round and keeps landing the same punch.

So you hire a sparring partner with one job: find the opening and throw through it every session. You get clipped in practice for weeks. Your guard moves up on its own. Fight night feels easier because the worst punch is old news.

That partner is the inner attack loop. It finds the perturbation that hurts most right now, and the outer training loop makes you defend it. Where the picture stops: a boxer tires and adapts between rounds, while this partner never tires and never gets creative beyond the small epsilon ball you allow it.

How It Actually Works

Adversarial training mixes clean inputs and attacker-made copies in the same update. Write the weights as θ\theta, one input as xx, its true label as yy, the small hostile shift as δ\delta (delta), the allowed size as ϵ\epsilon (epsilon), and the loss as LL. The goal is a min-max game: the attacker picks the worst δ\delta inside the budget, then you move θ\theta to lower the loss on that worst case.

min⁡θ[max⁡∥δ∥∞≤ϵloss(L(θ,x+δ,y))]\min_{\theta} \left[ \max_{\left\| \delta \right\|_{\infty} \le \epsilon} \text{loss}\left( L(\theta, x + \delta, y) \right) \right]

In plain words: θ\theta means every weight and bias you train, x+δx + \delta is the perturbed copy, the inner max⁡\max hunts the highest loss within the ϵ\epsilon box, and the outer min⁡\min updates the model against it. The ∞\infty subscript says no single input value may move by more than ϵ\epsilon.

The one-step attack you train against

The Fast Gradient Sign Method, called FGSM, takes one step in the direction that raises the loss fastest:

xadv=x+ϵ×sign(∇xL)x_{\text{adv}} = x + \epsilon \times \text{sign}\left( \nabla_{x} L \right)

Here xadvx_{\text{adv}} is the hostile copy, ∇xL\nabla_{x} L is the gradient of the loss with respect to the input pixels (not the weights), and sign\text{sign} keeps only each component's direction, +1+1 or −1-1. One step, one budget, no iteration.

Worked example with small numbers

Take a tiny logistic model with weights w=[1.0,1.0]w = [1.0, 1.0], meaning one weight per input, bias b=−0.25b = -0.25, input x=[0.2,0.1]x = [0.2, 0.1], and true label y=1y = 1. The logit zz is the raw score before the sigmoid, and pp is the predicted probability of label 1.

Clean pass: z=1.0×0.2+1.0×0.1−0.25=0.05z = 1.0 \times 0.2 + 1.0 \times 0.1 - 0.25 = 0.05. Then p=1/(1+e−0.05)≈0.512p = 1 / (1 + e^{-0.05}) \approx 0.512, just above the 0.5 line, so the model says label 1. The loss is −log⁡(0.512)≈0.668-\log(0.512) \approx 0.668.

Now attack with budget ϵ=0.1\epsilon = 0.1. The gradient of the loss with respect to xx is (p−y)×w=(0.512−1.0)×[1.0,1.0]=[−0.488,−0.488](p - y) \times w = (0.512 - 1.0) \times [1.0, 1.0] = [-0.488, -0.488]. Its sign is [−1,−1][-1, -1]. So xadv=[0.2−0.1,0.1−0.1]=[0.1,0.0]x_{\text{adv}} = [0.2 - 0.1, 0.1 - 0.1] = [0.1, 0.0].

Score it again: zadv=1.0×0.1+1.0×0.0−0.25=−0.15z_{\text{adv}} = 1.0 \times 0.1 + 1.0 \times 0.0 - 0.25 = -0.15. Then padv≈0.463p_{\text{adv}} \approx 0.463, now below 0.5, so the model says label 0 and gets it wrong. The loss climbs to −log⁡(0.463)≈0.771-\log(0.463) \approx 0.771. Same picture, same true label, shifted 0.1 per value, and the answer flips. The diagram above draws exactly this: +0.05+0.05 sliding to −0.15-0.15, and loss 0.6680.668 growing to 0.7710.771.

Training then does the outer step: it computes the loss on the clean [0.2,0.1][0.2, 0.1] and the hostile [0.1,0.0][0.1, 0.0] together, both tagged y=1y = 1, and moves ww and bb to pull both scores back above zero.

Why practice uses PGD instead of one step

FGSM takes one big stride. Projected Gradient Descent, called PGD, takes several small strides and clips back into the ϵ\epsilon box after each one. It finds a stronger δ\delta at the cost of more forward and backward passes per batch.

Models trained only on FGSM often learn to hide the gradient rather than move the boundary, then fall to PGD anyway. That is why PGD-made batches became the default opponent: beat the stronger search in training and the weaker one-step attacks usually fail too. The price is real, with slower steps and often a few points of clean accuracy traded for steadier behavior under attack. A sibling defense with a different promise, hiding single training rows instead of steadying the boundary, is differential privacy.

Skip this tool when your threat is plain noise, blur, or a new camera, not a crafted attack. For those shifts use wider data, augmentation, or recalibration, and save the PGD bill for models someone might actually prod on purpose.

Code

This snippet replays the hand calculation so you can check the flip yourself. It prints the clean and hostile probabilities plus both losses.

import math

def sigmoid(z: float) -> float:    return 1.0 / (1.0 + math.exp(-z))

def bce_loss(p: float, y: int) -> float:    return -(math.log(p) if y == 1 else math.log(1.0 - p))

w: list[float] = [1.0, 1.0]b: float = -0.25x: list[float] = [0.2, 0.1]y: int = 1eps: float = 0.1
z: float = w[0] * x[0] + w[1] * x[1] + bp: float = sigmoid(z)grad_sign: list[int] = [-1, -1]  # sign((p - y) * w) = sign([-0.488, -0.488])x_adv: list[float] = [x[0] + eps * grad_sign[0], x[1] + eps * grad_sign[1]]z_adv: float = w[0] * x_adv[0] + w[1] * x_adv[1] + bp_adv: float = sigmoid(z_adv)
print(round(p, 3))              # -> 0.512print(round(p_adv, 3))          # -> 0.463print(round(bce_loss(p, y), 3))      # -> 0.668print(round(bce_loss(p_adv, y), 3))  # -> 0.771

The printout matches the section above: 0.5120.512 against 0.4630.463, and 0.6680.668 against 0.7710.771. If your numbers differ, check the bias sign first, since −0.25-0.25 decides how close the clean point sits to the line.

Watch Out For

Training on FGSM only, then failing PGD in review

Symptom: training loss looks calm and one-step attacks fail, yet a 10-step PGD check flips the model. The model dulled the local gradient instead of moving the boundary, so the single step points nowhere useful while iteration walks around the trick.

Fix: train with PGD batches and judge with attacks held out from training, with more steps and a fresh random start. If PGD with extra steps still flips you, the boundary is still close, and no FGSM score overrules that.

Paying for epsilon you never meet in practice

Symptom: hostile accuracy rises a little while clean accuracy falls a lot, and each epoch takes several times longer. The ϵ\epsilon box is bigger than any shift your inputs will face, so training spends its budget smoothing directions that never matter.

Fix: match ϵ\epsilon to the threat you named before training, track clean and hostile accuracy side by side, and stop when clean keeps falling after hostile flattens. If no attacker exists, drop this method and spend the compute on data or tuning, as dropout and plain augmentation cost far less per epoch.

The Quick Version

  • Adversarial training feeds each batch twice: the clean input and its worst-case perturbed twin, both carrying the true label.
  • The setup is min-max: the inner search maximizes loss inside a small epsilon box, and the outer update minimizes it.
  • FGSM builds the twin in one signed gradient step, and the worked case moves 0.1 per value to flip 0.51 to 0.46.
  • PGD builds a stronger twin with several small steps plus clipping, which is why it is the usual training partner.
  • The cost is slower training and often lower clean accuracy in exchange for steadier answers under attack.
  • Use it when someone may craft inputs against you, not for plain noise or new cameras.