fast gradient sign method
In plain wordsA quick way to make an image that fools a neural network: nudge every pixel slightly in whichever direction most increases the network's error.
A one-step white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots. that generates an adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon. by moving each input component by a fixed amount ε in the direction of the sign of the loss gradientgradient, n.: A list of slopes showing which way, and how steeply, a quantity rises as each of its inputs changes. Training nudges an AI's settings in the opposite direction, downhill. with respect to the input. One gradient computation suffices, so it is cheap, and it was the basis of early adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.. Taking the sign maximizes the first-order (linear) increase in loss under an ℓ∞ constraint, which is why FGSM's success was read as evidence that networks behave too linearly. A targeted variant steps down the loss of a chosen wrong class; iterating small FGSM steps with projection gives the stronger projected gradient descent attackprojected gradient descent attack, n.: A stronger version of the quick pixel-nudging attack: take many small steps that each raise the model's error, pulling the image back after every step so the total change stays tiny..
FGSM at ε = 8/255 drops the undefended model's accuracy below 20%.
Origin
Introduced by Ian Goodfellow, Jonathon Shlens and Christian Szegedy in 'Explaining and Harnessing Adversarial Examples' (arXiv December 2014; ICLR 2015), which also proposed training on FGSM examples as a regularizer.
Usage note
Adversarial training with FGSM alone can suffer 'catastrophic overfitting': the model learns to resist the one-step attack while remaining vulnerable to multi-step ones.
Related
- adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon.
- projected gradient descent attackprojected gradient descent attack, n.: A stronger version of the quick pixel-nudging attack: take many small steps that each raise the model's error, pulling the image back after every step so the total change stays tiny.
- adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them.
- white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.
- gradientgradient, n.: A list of slopes showing which way, and how steeply, a quantity rises as each of its inputs changes. Training nudges an AI's settings in the opposite direction, downhill.
Sources
- Goodfellow, I. J., Shlens, J. & Szegedy, C. (2015). Explaining and Harnessing Adversarial Examples. ICLR 2015. arXiv:1412.6572.
Mentioned in
adversarial exampleadversarial example, n.: An input altered on purpose, often too slightly for people to notice, so that an AI model gets it badly wrong — like a panda photo that a classifier confidently calls a gibbon., adversarial trainingadversarial training, n.: Toughening an AI model by deliberately showing it attacks during training — the tricky inputs that fool it — along with the right responses, so it learns to resist them., Ian GoodfellowIan Goodfellow, prop. n.: An American computer scientist who invented generative adversarial networks, in which two neural networks compete to produce realistic fake images, and who pioneered research on fooling AI systems., Carlini–Wagner attackCarlini–Wagner attack, n.: A careful, optimization-based way of finding the smallest change to an input that fools a neural network; it broke several defenses that had looked secure., gradient maskinggradient masking, n.: When a defense against attacks only seems to work because it confuses the attacker's tools, not because the model is really any harder to fool., projected gradient descent attackprojected gradient descent attack, n.: A stronger version of the quick pixel-nudging attack: take many small steps that each raise the model's error, pulling the image back after every step so the total change stays tiny., white-box attackwhite-box attack, n.: An attack in which the attacker can see everything about an AI model, including its internal settings, and uses that knowledge to find its weak spots.