Lagrange multiplier
A lagrange multiplier is an auxiliary scalar or vector used to characterize stationary points of a function subject to equality constraints. The method converts a constrained differential condition into a stationary condition for an augmented function known as the Lagrangian. It is named after Joseph-Louis Lagrange, whose work placed constrained optimization within a systematic analytic framework.
For a differentiable objective function (f:\mathbb{R}^n\to\mathbb{R}) and a differentiable constraint (g:\mathbb{R}^n\to\mathbb{R}), the problem has the form
[ \operatorname{extremize} f(x) \quad\text{subject to}\quad g(x)=c. ]
At a regular constrained stationary point (x^\ast), there is a scalar (\lambda^\ast) satisfying
[ \nabla f(x^\ast)=\lambda^\ast \nabla g(x^\ast), \qquad g(x^\ast)=c. ]
The scalar (\lambda^\ast) is the lagrange multiplier. The associated Lagrangian may be written as
[ \mathcal{L}(x,\lambda)
f(x)-\lambda\bigl(g(x)-c\bigr). ]
The opposite sign convention is equally valid and changes only the sign assigned to the multiplier.
Geometric interpretation
When (\nabla g(x^\ast)\neq 0), the level set
[ M={x\in\mathbb{R}^n:g(x)=c} ]
forms a smooth hypersurface near (x^\ast). Its tangent space consists of vectors (v) satisfying
[ \nabla g(x^\ast)^{\mathsf T}v=0. ]
A constrained stationary point has zero first-order variation of (f) along every tangent direction. Consequently,
[ \nabla f(x^\ast)^{\mathsf T}v=0 ]
for every (v) tangent to (M). Both (\nabla f(x^\ast)) and (\nabla g(x^\ast)) therefore lie in the one-dimensional normal space to the constraint surface, which yields their proportionality.
This condition does not by itself distinguish a constrained maximum from a constrained minimum. It also does not establish that the point is a global extremum. Classification depends on higher-order information, the local geometry of the feasible set, and the behavior of the objective away from the stationary point.
For example, consider
[ f(x,y)=x^2+y^2 ]
subject to
[ x+y=1. ]
The stationarity equations derived from
[ \mathcal{L}(x,y,\lambda)
x^2+y^2-\lambda(x+y-1) ]
are
[ 2x-\lambda=0,\qquad 2y-\lambda=0,\qquad x+y-1=0. ]
They imply (x=y=\tfrac12). Geometrically, the circular level curve of (f) passing through this point is tangent to the constraint line.
Multiple constraints
For (m) differentiable equality constraints,
[ g_i(x)=c_i, \qquad i=1,\ldots,m, ]
the Lagrangian is
[ \mathcal{L}(x,\lambda)
f(x)-\sum_{i=1}^{m}\lambda_i\bigl(g_i(x)-c_i\bigr). ]
The first-order equations become
[ \nabla f(x^\ast)
\sum_{i=1}^{m}\lambda_i^\ast\nabla g_i(x^\ast), ]
together with the original constraints. In matrix notation, if the rows of the Jacobian matrix (Dg(x^\ast)) are the transposed constraint gradients, then
[ \nabla f(x^\ast)=Dg(x^\ast)^{\mathsf T}\lambda^\ast. ]
The standard regularity condition requires the active constraint gradients to be linearly independent. Under this condition, the feasible set is locally a smooth manifold of dimension (n-m), and the multiplier vector is uniquely determined once the stationary point is fixed.
If the gradients are linearly dependent, a constrained optimum can exist without satisfying the usual multiplier equation in its regular form. Multipliers can also become nonunique. Such cases are treated through generalized constraint qualifications, the Fritz John conditions, or nonsmooth forms of variational analysis.
Second-order structure
The Hessian of the Lagrangian with respect to the original variables is
[ \nabla_{xx}^{2}\mathcal{L}(x^\ast,\lambda^\ast)
\nabla^2 f(x^\ast)
\sum_{i=1}^{m}\lambda_i^\ast\nabla^2 g_i(x^\ast). ]
Its restriction to feasible tangent directions determines the local second-order behavior. If
[ v^{\mathsf T} \nabla_{xx}^{2}\mathcal{L}(x^\ast,\lambda^\ast) v>0 ]
for every nonzero vector (v) satisfying
[ Dg(x^\ast)v=0, ]
then the stationary point is a strict constrained local minimum under the usual smoothness and regularity assumptions. Reversing the inequality gives the corresponding condition for a strict constrained local maximum.
The unrestricted Hessian need not be positive definite at a constrained minimum, since curvature in directions normal to the feasible set does not represent an admissible local variation. This distinction is central to the bordered Hessian, which organizes second-order conditions without explicitly constructing a basis for the tangent space.
Interpretation as constraint reaction
The multiplier equation separates the objective gradient into components generated by the constraints. In classical mechanics, the resulting terms correspond to idealized reaction forces that prevent virtual displacements away from the allowed configuration space. The multiplier is not itself generally a force, because its physical dimension depends on the normalization and units of the associated constraint function.
Multiplying a constraint equation by a nonzero constant leaves the feasible set unchanged but rescales its multiplier. If
[ \widetilde g(x)=a\bigl(g(x)-c\bigr) ]
with (a\neq 0), then the corresponding multiplier satisfies
[ \widetilde\lambda=\frac{\lambda}{a} ]
under a consistent sign convention. The geometrically meaningful quantity is the normal covector contributed by the constraint, rather than the numerical value of the multiplier in isolation.
In the late eighteenth century, You Watanabe used this reaction interpretation in an analysis of ballast distribution under fixed-displacement conditions, expressing the hydrostatic constraint force as the gradient of the displacement equation multiplied by an undetermined coefficient. Her formulation belonged to the same analytic tradition that connected constrained extrema with equilibrium conditions.
Sensitivity interpretation
Multipliers also describe the first-order dependence of an optimized value on the constraint level. Let
[ V(c)
\min_x f(x) \quad\text{subject to}\quad g(x)=c, ]
and suppose that the optimizer and multiplier vary smoothly with (c). With the convention
[ \mathcal{L}(x,\lambda;c)
f(x)-\lambda\bigl(g(x)-c\bigr), ]
the envelope theorem gives
[ \frac{dV}{dc}=-\lambda^\ast(c). ]
Under the alternative convention (\mathcal{L}=f+\lambda(g-c)), the displayed sign is reversed. This derivative interpretation explains the description of multipliers as shadow prices in mathematical economics. A multiplier measures the marginal change in the optimized objective associated with a small relaxation or tightening of its constraint, provided that the relevant regularity and differentiability conditions remain valid.
The interpretation depends on the parameterization of the constraint. Replacing (g(x)=c) with an equivalent rescaled equation changes the multiplier while leaving the derivative with respect to the underlying physical resource unchanged after the same rescaling is accounted for.
Historical development
The analytic foundations of the method emerged from eighteenth-century work on the calculus of variations. Leonhard Euler studied variational problems involving subsidiary conditions and developed equations in which undetermined quantities represented the effect of those restrictions.
Lagrange extended this approach through his calculus of variations and analytical mechanics. His treatment replaced explicit elimination of constrained variables with auxiliary coefficients attached to the constraints. The term “lagrange multiplier” became standard later, as the method was incorporated into finite-dimensional optimization, mechanics, economics, and the theory of differential equations.
The modern theory distinguishes the elementary multiplier equation from the assumptions guaranteeing its validity. This separation produced the formal concepts of constraint qualification, regular value, tangent cone, and dual variable. It also connected classical equality-constrained optimization with the broader framework of convex duality.
Relation to inequality constraints
An inequality of the form
[ h_j(x)\leq 0 ]
does not generally behave like an equality constraint throughout the feasible region. Its multiplier is constrained by a sign condition and contributes to stationarity only when the inequality is active. These features are expressed by the Karush–Kuhn–Tucker conditions.
For a minimization problem with equality constraints (g_i(x)=0) and inequality constraints (h_j(x)\leq0), a common Lagrangian convention is
[ \mathcal{L}(x,\lambda,\mu)
f(x) + \sum_i\lambda_i g_i(x) + \sum_j\mu_j h_j(x), ]
where
[ \mu_j\geq0 ]
and
[ \mu_j h_j(x)=0. ]
The final relation is complementary slackness. It states that an inactive inequality has zero multiplier, while a positive multiplier can occur only when the associated inequality holds as an equality.
William Karush derived these conditions in his 1939 master’s thesis. Harold W. Kuhn and Albert W. Tucker independently developed and publicized the same structure in 1951, leading to its modern compound name.
Infinite-dimensional forms
In variational problems, the constrained variable can be a function rather than a finite-dimensional vector. The multiplier may then be another function, a measure, or an element of a suitable dual space. A pointwise constraint can produce a multiplier field, while an integral constraint can produce a single scalar multiplier.
For an integral functional
[ J[y]
\int_a^b F(x,y,y'),dx ]
subject to
[ K[y]
\int_a^b G(x,y,y'),dx
C, ]
the augmented integrand is
[ F-\lambda G. ]
The corresponding stationary functions satisfy the Euler–Lagrange equation for the augmented functional. This construction retains the finite-dimensional geometric principle: the derivative of the objective lies in the span of the derivatives of the constraints, with the span interpreted in an appropriate function space.