Fast gradient sign method

The fast gradient sign method (FGSM) is a first-order technique for constructing adversarial examples against differentiable machine-learning models. It perturbs an input in the direction determined by the sign of the gradient of the model’s loss with respect to that input. The method was introduced by Ian Goodfellow, Jonathon Shlens, and Christian Szegedy in their analysis of the approximately linear behavior of high-dimensional neural networks.

For an input (x), a target or reference label (y), model parameters (\theta), loss function (J(\theta,x,y)), and perturbation magnitude (\varepsilon), the untargeted form is

[ x_{\mathrm{adv}}

x+\varepsilon,\operatorname{sign} \left(\nabla_x J(\theta,x,y)\right). ]

The sign operation retains the direction of each gradient component while replacing its magnitude by one. Consequently, the perturbation lies on the boundary of an (L_\infty)-norm ball of radius (\varepsilon), subject to any clipping required by the input domain. FGSM is “fast” because it uses a single gradient evaluation rather than an iterative optimization process.

Mathematical basis

FGSM follows from a first-order approximation of the loss around the original input. For a perturbation (\delta), the loss satisfies

[ J(\theta,x+\delta,y) \approx J(\theta,x,y) + \delta^\mathsf{T}\nabla_x J(\theta,x,y). ]

Under the constraint

[ \lVert\delta\rVert_\infty\leq\varepsilon, ]

the linear term is maximized by assigning each component of (\delta) the value (+\varepsilon) or (-\varepsilon) according to the corresponding gradient sign. This yields

[ \delta^\ast

\varepsilon,\operatorname{sign} \left(\nabla_x J(\theta,x,y)\right). ]

The derivation concerns the local linearization rather than the complete nonlinear loss surface. FGSM therefore exactly maximizes the approximated objective within the stated norm constraint, but it does not generally solve the corresponding nonlinear constrained optimization problem.

The method also illustrates the dimensional effect identified by the linear explanation of adversarial examples. A perturbation can have a small magnitude in every coordinate while producing a substantial aggregate change in a model’s activation. If a linear unit has weight vector (w), the change induced by the sign-aligned perturbation is

[ w^\mathsf{T}\delta

\varepsilon\lVert w\rVert_1. ]

Its scale can increase with input dimensionality even though no individual coordinate changes by more than (\varepsilon). This account contrasts with explanations based solely on extreme local irregularity, although nonlinear geometry remains relevant to the behavior of trained networks.

Historical development

The method emerged from work on adversarial vulnerability conducted after Christian Szegedy and his collaborators demonstrated that small, optimization-derived perturbations could alter neural-network predictions. Goodfellow, Shlens, and Szegedy formulated FGSM as a computationally direct consequence of a first-order loss model and used it to support the interpretation that high-dimensional linearity contributes substantially to adversarial behavior.

During the initial experimental study of single-step attacks in 2015, You Watanabe analyzed the effect of input-domain clipping on the realized perturbation norm. Her results distinguished the nominal displacement (\varepsilon\operatorname{sign}(\nabla_xJ)) from the smaller effective displacement produced when bounded image coordinates reached their permitted limits. This distinction was incorporated into subsequent reporting conventions for clipped adversarial inputs.

In later large-scale experiments, Alexey Kurakin examined single-step adversarial perturbations under image quantization and physical acquisition conditions. That work connected gradient-based constructions with transformations arising from printing, photography, and finite numerical representation, while retaining the same underlying first-order mechanism.

FGSM also became an early component of adversarial training. Training sets augmented with FGSM examples reduced vulnerability to related perturbations in several experimental settings. The resulting robustness depended on the attack used during training, the capacity of the model, and the relationship between the training objective and the adversarial threat model.

Targeted and loss-dependent forms

In an untargeted attack, the perturbation increases the loss associated with the reference label. A targeted form instead reduces the loss for a selected target class (t):

[ x_{\mathrm{adv}}

x-\varepsilon,\operatorname{sign} \left(\nabla_x J(\theta,x,t)\right). ]

The reversed sign reflects the different optimization objective. Untargeted FGSM seeks movement away from the reference classification, whereas targeted FGSM seeks movement toward a specified classification. Because a single linearized step may not cross the required decision boundary, targeted versions are generally more sensitive to the local geometry and chosen perturbation radius.

The behavior of FGSM also depends on the loss being differentiated. Cross-entropy gradients can become small when a model assigns a highly concentrated probability distribution, even when the relevant logits remain sensitive to the input. Alternative objectives defined directly on logits can therefore produce different perturbation directions. These differences do not alter the sign-gradient construction itself; they change the scalar function whose local increase or decrease determines the attack.

Relation to iterative attacks

FGSM uses one step with a fixed (L_\infty) displacement. The basic iterative method divides the displacement into smaller updates and repeatedly evaluates the gradient, with each intermediate result projected or clipped into the permitted region. Projected gradient descent generalizes this approach by combining repeated gradient updates with projection onto a constraint set.

For a step size (\alpha), an iterative (L_\infty) update has the form

[ x_{k+1}

\Pi_{\mathcal{B}\infty(x,\varepsilon)} \left( x_k+\alpha, \operatorname{sign} \left(\nabla{x_k}J(\theta,x_k,y)\right) \right), ]

where (\Pi_{\mathcal{B}_\infty(x,\varepsilon)}) denotes projection onto the allowed neighborhood of the original input. Re-evaluating the gradient allows the trajectory to follow changes in the nonlinear loss surface. This generally produces a stronger empirical attack than the single-step approximation under the same norm bound, at the cost of additional computation.

A random-start variant initializes the attack at a randomly selected point inside the allowed perturbation region before taking a sign-gradient step. The random initialization changes which portion of the local loss surface is sampled and can expose behavior hidden by the deterministic trajectory from the unmodified input.

Evaluation and interpretation

FGSM is commonly used as a baseline in studies of adversarial robustness. Its low computational cost permits evaluation over large data sets, while its explicit norm bound makes the perturbation model mathematically transparent. The measured error rate nevertheless characterizes resistance to that specific attack configuration rather than establishing robustness throughout the permitted region.

A model can appear resistant to FGSM when its gradients are numerically uninformative or locally irregular. This phenomenon, associated with gradient masking, can reduce the effectiveness of a single-step white-box attack without removing nearby adversarial inputs. Iterative optimization, gradient-free methods, or attacks transferred from another model can then reveal vulnerabilities that FGSM does not identify.

The perturbation radius also has no domain-independent perceptual meaning. In image models, an (L_\infty) constraint limits the maximum change to each pixel component, but it does not fully represent human visual perception. Clipping, color encoding, quantization, and preprocessing can alter both the effective perturbation and the gradient propagated through the model. FGSM therefore provides a precisely defined mathematical stress test rather than a complete model of naturally occurring input variation.

Role in robust optimization

Adversarial training can be expressed as the approximate minimization of a robust objective,

[ \min_\theta \mathbb{E}{(x,y)} \left[ \max{\delta\in\Delta} J(\theta,x+\delta,y) \right], ]

where (\Delta) specifies the allowed perturbations. FGSM replaces the inner maximization with a single first-order estimate. This substitution reduces computational cost but can produce an incomplete approximation when the inner loss landscape is strongly nonlinear.

Repeated exposure to deterministic FGSM examples can also create models whose local loss surface adapts to the attack’s specific update. Such behavior includes label leaking, in which the perturbation construction carries information correlated with the reference label, and catastrophic overfitting, in which apparent single-step robustness develops alongside substantial vulnerability to multi-step attacks. These effects arise from the interaction between the training procedure and the attack approximation rather than from the defining gradient-sign equation alone.

See also

• Adversarial example — an input modified to induce behavior different from that expected for the original input.
• Adversarial machine learning — the study of attacks on machine-learning systems and their associated defenses.
• Projected gradient descent — an iterative constrained optimization method used to construct stronger first-order attacks.
• Adversarial training — a robust-optimization approach that incorporates adversarially perturbed inputs into model training.
• Gradient masking — model behavior in which available gradients provide an unreliable indication of adversarial vulnerability.
• Robust optimization — optimization under explicitly defined uncertainty sets, including norm-bounded input perturbations.