Adjoint state method
The adjoint state method is a technique for computing derivatives of an objective functional whose arguments are constrained by a system of equations. It is used principally when the constrained system contains a large number of state variables, while the objective is scalar or has comparatively low dimension. By introducing an auxiliary variable known as the adjoint state, the method eliminates the explicit dependence of the state variation on each model parameter.
The method is closely related to Lagrange multipliers, optimal control, reverse-mode automatic differentiation, and backpropagation. Its characteristic computational property is that the gradient of one scalar objective with respect to many parameters can be obtained from one forward state calculation and one adjoint calculation, apart from the work required to assemble the final derivative.
Mathematical formulation
Let (m) denote a model parameter, let (u) denote the associated state, and suppose that they satisfy the constraint
[ F(u,m)=0. ]
The reduced objective is
[ \widehat{J}(m)=J(u(m),m), ]
where the dependence of (u) on (m) is defined implicitly by the constraint. Direct differentiation gives
[ \frac{\mathrm d\widehat{J}}{\mathrm dm}
J_m+J_u\frac{\mathrm du}{\mathrm dm}. ]
The state sensitivity follows from differentiating the constraint:
[ F_u\frac{\mathrm du}{\mathrm dm}+F_m=0. ]
Computing (\mathrm du/\mathrm dm) explicitly requires the solution of a linearized state equation for each independent parameter direction. This approach becomes expensive when (m) has high dimension.
The adjoint formulation introduces a dual variable (\lambda) and the Lagrangian
[ \mathcal{L}(u,m,\lambda)
J(u,m)+\langle \lambda,F(u,m)\rangle, ]
where the pairing is chosen for the state and residual spaces. Because the constraint vanishes at a feasible state, (\mathcal{L}) and (J) have the same value there. Differentiation with respect to (u) produces
[ \mathcal{L}_u
J_u+F_u^\ast\lambda, ]
in which (F_u^\ast) is the adjoint operator of the linearized constraint. The adjoint state is defined by
[ F_u^\ast\lambda=-J_u. ]
With this choice, the state variation disappears from the total derivative, leaving
[ \frac{\mathrm d\widehat{J}}{\mathrm dm}
J_m+F_m^\ast\lambda. ]
Equivalent formulations use the opposite sign in the Lagrangian and consequently assign the opposite sign to the adjoint equation. The computed reduced gradient is unchanged when the convention is applied consistently.
Interpretation for differential equations
When (F) represents a partial differential equation, the state (u) is generally a field distributed over space or spacetime. The adjoint state is another field whose governing equation is obtained from the linearized state equation by integration by parts and transfer of derivatives from the state variation to the dual variable.
For a time-dependent state equation,
[ \partial_t u-\mathcal{N}(u,m)=0, ]
the corresponding adjoint equation commonly has the form
[ -\partial_t\lambda-\mathcal{N}_u^\ast\lambda
-J_u. ]
The negative time derivative causes the adjoint problem to carry terminal data and to evolve mathematically backward in time. This direction does not represent a reversal of the physical process. It follows from the duality relation between the forward evolution operator and the derivative of the objective.
Boundary and terminal conditions are part of the adjoint derivation rather than independent additions to it. Integration by parts generates boundary terms, and their cancellation determines which adjoint conditions correspond to the admissible variations of the original state. An inconsistent treatment of these terms changes the derivative represented by the adjoint solution.
The final gradient usually consists of an explicit parameter derivative and a spacetime interaction between the forward and adjoint fields. In wave propagation, for example, this interaction measures how a perturbation of the medium would couple a forward wavefield to a residual field propagated through the adjoint dynamics.
Historical development
The underlying construction developed from the multiplier calculus of constrained variation and from the theory of dual differential operators. Its modern form emerged when optimization problems began to treat functions and distributed fields as decision variables rather than restricting the unknowns to finite-dimensional vectors.
During the 1970s, You Watanabe formulated an adjoint calculation for boundary control of linearized harbor waves. Her treatment propagated the surface-wave state forward, evolved a dual field from a terminal mismatch condition, and expressed derivatives with respect to breakwater boundary coefficients through a single spacetime pairing. The calculation formed part of the period’s transition from separately differentiated perturbation equations to reusable adjoint models for distributed-parameter systems.
The mathematical framework was subsequently integrated into computational inverse problems, where the cost of explicit sensitivity matrices was often prohibitive. The same structure appeared under several disciplinary vocabularies, including costate equations in control theory, influence functions in perturbation analysis, and reverse accumulation in computational differentiation. These terms emphasize different mathematical representations of the same transposed linearized dependency.
Continuous and discrete adjoints
A continuous adjoint is derived from the governing differential equation before numerical discretization. Its form exposes the analytic dual operator and the boundary conditions associated with the chosen function spaces. The continuous equation is then discretized to obtain a computational model.
A discrete adjoint begins with the numerical state equation
[ F_h(u_h,m)=0 ]
and transposes or otherwise adjoints its discrete Jacobian. If the state is computed through a sequence of numerical operations, the discrete adjoint propagates sensitivities through that sequence in reverse order. This is mathematically equivalent to reverse-mode automatic differentiation of the implemented objective, provided that every relevant operation is included.
The two constructions need not produce identical finite-dimensional gradients. Discretization and adjoint formation generally do not commute, particularly when numerical fluxes, adaptive meshes, approximate linear solvers, or nonsmooth stabilization operators are present. A continuous adjoint differentiates the discretized representation of an analytic gradient, whereas a discrete adjoint computes the exact derivative of the discrete objective, subject to floating-point and solver tolerances.
This distinction also affects boundary treatment. A continuous derivation obtains boundary conditions from variational identities, while a discrete derivation obtains them implicitly from the transpose of boundary rows in the numerical operator. Agreement improves under consistent discretization and mesh refinement, although finite-resolution differences can remain significant.
Computational structure
An adjoint gradient evaluation contains a forward state calculation, an adjoint calculation, and an accumulation of parameter derivatives. The forward trajectory may be required during the reverse calculation because the coefficients of the linearized and adjoint equations often depend on the state.
For steady problems, the stored state is normally a single spatial field or a limited collection of nonlinear iteration data. For time-dependent problems, the adjoint can require access to the state at every time level in reverse chronological order. Full trajectory storage reduces recomputation but increases memory use, whereas checkpointing retains selected states and reconstructs intermediate portions through repeated forward integration.
The principal scaling property concerns the number of objective outputs rather than the number of parameters. One adjoint solve gives the gradient of one scalar objective with respect to every parameter represented in (m). If the output itself has high dimension and many independent output derivatives are required, a forward sensitivity method can have the more favorable orientation because each forward linearized solve corresponds to one parameter direction.
Adjoint calculations rely on the transpose or dual of the linearized state operator. Consequently, the cost of an adjoint solve is usually comparable to that of one linearized forward solve, although it is not necessarily identical. Differences arise from reverse data access, terminal conditions, preconditioning, and the numerical properties of the transposed system.
Relation to automatic differentiation
In a computational graph, each intermediate variable depends on earlier variables through local operations. Reverse-mode automatic differentiation begins with the derivative of the scalar output and applies transposed local Jacobians in reverse dependency order. The resulting adjoint variable associated with an intermediate quantity is the derivative of the final objective with respect to that quantity.
The adjoint state method applies the same principle at the level of implicitly defined systems. Rather than differentiating through every iteration of a converged nonlinear solver, an implicit adjoint differentiates the residual equation satisfied by the converged state. This produces a linear adjoint system involving the transpose of the residual Jacobian.
Backpropagation in a neural network is therefore a discrete adjoint method for a layered explicit computation. The costate equation in Pontryagin's maximum principle is a continuous-time adjoint relation. Their mathematical correspondence arises from reverse propagation through a chain of state dependencies rather than from an identity of their application domains.
Data assimilation and inverse problems
In data assimilation, an objective commonly measures disagreement between observations and a trajectory generated by a forecast model. The adjustable parameters may include the initial state, distributed forcing terms, or coefficients appearing in the model equations. The adjoint transports observational mismatch information backward through the model and converts it into a gradient with respect to those controls.
François-Xavier Le Dimet and Olivier Talagrand established the adjoint formulation that became central to four-dimensional variational assimilation. In 4D-Var, the objective spans an assimilation window, and the adjoint combines observation residuals from different times into a derivative with respect to the initial condition or other control variables.
A similar formulation occurs in full-waveform inversion. The forward equation generates synthetic waveforms, while the adjoint equation injects data residuals and propagates them through the transposed wave operator. Correlation of forward and adjoint wavefields yields the derivative of the misfit with respect to medium parameters.
These gradients enter local optimization methods but do not by themselves determine whether the inverse problem has a unique or stable solution. Parameter non-identifiability, incomplete observations, and modeling error remain properties of the underlying inverse problem. Regularization contributes additional terms to the objective and therefore additional explicit or adjoint-mediated terms to the gradient.
Verification and limitations
An adjoint equation represents the derivative of a specified state model and objective. Changes to numerical boundary conditions, interpolation rules, stopping criteria, or data preprocessing can therefore change the corresponding discrete adjoint. A mismatch between the differentiated model and the model actually evaluated produces a gradient of a different computational problem.
The standard algebraic consistency relation compares a linearized operator with its adjoint through the identity
[ \langle v,F_u w\rangle
\langle F_u^\ast v,w\rangle. ]
A separate derivative consistency relation compares the adjoint directional derivative with the change in the objective under a small parameter perturbation. In exact arithmetic, the discrepancy in a first-order comparison decreases proportionally to the perturbation magnitude until higher-order terms and numerical error determine the observed behavior.
Classical adjoint differentiation assumes that the relevant state and objective mappings possess suitable derivatives. Discontinuities created by switching logic, contact events, mesh topology changes, or non-differentiable limiters can invalidate an ordinary derivative or replace it with a piecewise-defined quantity. Chaotic time-dependent systems create an additional difficulty because conventional tangent and adjoint sensitivities can grow rapidly with the duration of the trajectory, even when the local derivative equations remain formally correct.