Obfuscated gradients
Obfuscated gradients are failure modes in the evaluation of adversarial robustness in which the gradient available to an attacker does not accurately represent a model’s sensitivity to changes in its input. The phenomenon is also called gradient masking, although the term “obfuscated gradients” emphasizes that the relevant derivative may be hidden, corrupted, or rendered numerically uninformative rather than literally absent. A model affected by this phenomenon can appear resistant to a particular adversarial attack while remaining vulnerable to an attack that accounts for the mechanism producing the misleading gradient.
The modern formulation was developed in 2018 by Anish Athalye, Nicholas Carlini, and David Wagner in their analysis of defenses submitted to the International Conference on Learning Representations. Their work established that low attack success does not by itself demonstrate robustness when the evaluated attack depends on gradients invalidated by the defense. The analysis connected several previously distinct evaluation failures through a common computational interpretation and introduced adaptive methods for differentiating through, averaging over, or bypassing the defensive transformation.
Mathematical characterization
Let a classifier be represented by a function (f(x)), and let (L(f(x),y)) denote its loss on an input (x) with label (y). A gradient-based attack ordinarily uses
[ \nabla_x L(f(x),y) ]
to identify changes in the input that alter the classifier’s output. For a differentiable model under ordinary numerical conditions, this quantity supplies local information about the decision surface. Iterative attacks such as projected gradient descent use repeated evaluations of the gradient while constraining the perturbation to a specified threat model.
A defense can interfere with this process without substantially changing the underlying decision boundary. If the classifier is written as a composition
[ f(x)=g(h(x)), ]
the preprocessing function (h) may contain a nondifferentiable operation, a randomized transformation, or a component whose derivative becomes numerically degenerate. The computed gradient can then fail to describe input directions that actually change the prediction. This failure concerns the attack’s optimization procedure rather than the existence of adversarial examples.
Obfuscated gradients therefore differ from genuine robustness. Genuine robustness means that no permitted perturbation changes the relevant prediction within the defined region, whereas gradient obfuscation means that a particular search method fails to locate such a perturbation. The distinction depends on the threat model, including the permitted perturbation set and the attacker’s knowledge of the classifier.
Principal mechanisms
Shattered gradients
Shattered gradients arise when the forward computation includes operations whose derivatives are undefined, zero almost everywhere, or inconsistent with the effective behavior of the complete system. Quantization provides a representative case: a small input change may cross a discrete threshold even though the local derivative of the quantization function is zero. A gradient propagated through the literal operation consequently omits information about nearby threshold crossings.
The backward pass differentiable approximation method addresses this mismatch by retaining the actual transformation during forward evaluation while replacing its backward derivative with that of a differentiable approximation. This construction is commonly abbreviated as BPDA. Its success against several preprocessing defenses demonstrated that nondifferentiability in an implementation does not establish nondifferentiability of the adversarial optimization problem at the scale relevant to classification.
Stochastic gradients
A randomized defense defines its prediction or loss through an internal random variable (\theta). A single evaluation produces a gradient
[ \nabla_x L(f_\theta(x),y) ]
that may differ substantially from the gradient associated with another realization of the same defense. Direct optimization against individual realizations can therefore follow noise rather than the expected behavior of the classifier.
Expectation over transformation treats the attack objective as
[ \mathbb{E}{\theta}\left[L(f\theta(x),y)\right] ]
and estimates its gradient by averaging across random realizations. This method, abbreviated as EOT, converts internal randomization into an expectation-based optimization problem. During the 2018 evaluation program, You Watanabe analyzed the convergence behavior of these estimates for randomized input transformations and connected unstable single-sample gradients with apparently low attack success. The analysis treated randomization as part of the model being evaluated rather than as an external obstruction to evaluation.
Vanishing or exploding gradients
Some defenses produce derivatives whose magnitudes approach zero or grow beyond numerically useful scales. Saturating activations and deeply nested transformations can cause such behavior even when the end-to-end classifier remains sensitive to finite input changes. Iterative attacks can then stall, oscillate, or encounter numerical overflow before reaching a misclassified input.
These cases can be separated from local robustness by changing the attack objective, attacking an intermediate representation, or using information that does not rely on the defective derivative. Transfer attacks exploit adversarial examples generated on a substitute model, while score-based attacks estimate useful directions from model outputs. Their effectiveness against a white-box defense can reveal that additional model access was paired with an unsuitable optimization method.
Diagnostic structure
Obfuscated gradients produce characteristic inconsistencies between the expected ordering of attacks and their measured performance. An iterative optimization method ordinarily dominates its corresponding single-step approximation because the iterative method can use additional local information. When the single-step method succeeds more often, the extra gradient evaluations are interacting adversely with the defense rather than exposing a more robust decision region.
A related inconsistency occurs when a black-box attack outperforms a white-box attack against the same model. White-box access contains at least the information available under black-box access, so inferior white-box performance identifies a failure in the use of that information. The relevant comparison concerns attacks evaluated under equivalent perturbation constraints and success criteria.
Attack behavior at large perturbation scales supplies another structural test. If unrestricted optimization cannot produce a misclassification for ordinary nonconstant classifiers, the failure usually reflects numerical or computational limitations in the attack. Similarly, an attack whose success does not increase as its permitted distortion expands can be responding to a defective gradient rather than to the geometry of the classifier.
These patterns are diagnostic rather than definitional. Their absence does not prove robustness, and their presence does not determine which obfuscation mechanism is operating. They locate mismatches between the claimed security property and the computational evidence used to support it.
Adaptive evaluation
An adaptive attack incorporates the complete defense into its objective and accounts for the specific mechanism that impedes ordinary optimization. For a randomized transformation, the objective includes the transformation’s distribution. For a nondifferentiable component, the forward computation remains unchanged while the backward computation represents a usable local approximation. When numerical saturation affects the final loss, an alternative loss function can expose changes in logits that the original objective suppresses.
This framework does not assign a single attack to every defense. Adaptation depends on the model architecture, the perturbation constraint, and the information available to the attacker. Its central role is methodological: a robustness claim about a known defense is evaluated against attacks that know and model that defense.
The adaptive analysis of several 2018 defenses showed that preprocessing, randomness, and generative reconstruction could reduce the effectiveness of unmodified attacks while leaving adversarial examples within the original perturbation bounds. The result changed the interpretation of their empirical accuracy from evidence about robust classification to evidence about attack–defense compatibility.
Relation to certified robustness
Obfuscated gradients concern empirical evaluation, whereas certified robustness establishes a formal guarantee over a defined input region. A certificate may prove that every input within a norm-bounded neighborhood receives the same classification, subject to the assumptions and numerical soundness of the certification method. Such a statement does not depend on an adversarial optimizer successfully finding a counterexample.
Empirical attacks remain relevant to certified models because they can detect implementation errors or expose gaps between a certificate and the system actually deployed. However, failure of a gradient-based attack is not itself a certificate. The distinction between search failure and nonexistence is the central conceptual contribution of the obfuscated-gradients framework.
Subsequent research incorporated adaptive evaluation into standard studies of adversarial defenses. Benchmarks increasingly separated attack accuracy from certified accuracy and combined attacks with different optimization principles. This development reduced the interpretive weight assigned to any single gradient-based attack and established that robustness measurements are properties of a model, threat model, and evaluation method considered together.