Convex function

A real-valued function is a convex function when its value at every weighted average of two points does not exceed the corresponding weighted average of its values. Convex functions provide the analytic counterpart of convex sets, since their defining inequality expresses the absence of upward curvature along every line segment in the domain. They occupy a central position in mathematical analysis, optimization, and probability theory.

For a convex subset (C) of a real vector space, a function (f:C\to\mathbb R) is convex if

[ f\bigl(\lambda x+(1-\lambda)y\bigr) \leq \lambda f(x)+(1-\lambda)f(y) ]

for all (x,y\in C) and every (\lambda\in[0,1]). Geometrically, the graph of (f) between two points lies at or below the chord joining those points. Despite the terminology, the graph itself need not be a convex set.

Definition and principal variants

The domain must be convex because the point (\lambda x+(1-\lambda)y) occurring in the defining inequality must remain within the domain. A function defined on a nonconvex set can instead be studied through its restrictions to convex subsets, although it is not ordinarily called convex on the original domain.

A function is strictly convex if the defining inequality is strict whenever (x\neq y) and (0<\lambda<1):

[ f\bigl(\lambda x+(1-\lambda)y\bigr) < \lambda f(x)+(1-\lambda)f(y). ]

Strict convexity prevents the graph from containing a nontrivial affine segment. Consequently, a strictly convex function has at most one global minimizer on a convex domain, provided that such a minimizer exists. The converse does not hold, since uniqueness of a particular minimum does not control the function's curvature elsewhere.

In a normed vector space, (f) is (m)-strongly convex for (m>0) when

[ f\bigl(\lambda x+(1-\lambda)y\bigr) \leq \lambda f(x)+(1-\lambda)f(y) -\frac{m}{2}\lambda(1-\lambda)\lVert x-y\rVert^2. ]

This property states that (f) has at least a fixed quadratic amount of curvature relative to the chosen norm. Strong convexity implies strict convexity, while strict convexity alone does not imply any positive strong-convexity parameter.

A function (f) is concave precisely when (-f) is convex. Results about convex functions therefore have corresponding formulations for concave functions, with the directions of the relevant inequalities reversed.

Epigraph and geometric characterization

The epigraph of a function (f:C\to\mathbb R) is

[ \operatorname{epi} f

{(x,t)\in C\times\mathbb R:t\geq f(x)}. ]

A function is convex if and only if its epigraph is a convex set. This equivalence translates the functional inequality into a geometric statement: whenever two points lie on or above the graph, the entire line segment between them also lies on or above the graph.

The epigraph formulation extends naturally to functions taking values in the extended real number system. In convex analysis, a function

[ f:X\to(-\infty,+\infty] ]

may assign (+\infty) outside the region where it is effectively defined. Its effective domain is

[ \operatorname{dom} f={x\in X:f(x)<+\infty}. ]

Under this convention, constraints can be represented by an indicator function that equals zero on a convex feasible set and (+\infty) outside it. The resulting extended-real function is convex exactly when the feasible set is convex.

A convex function is lower semicontinuous if and only if its epigraph is closed, subject to the standard product topology. Closed convex functions are therefore the functional analogues of closed convex sets and behave consistently under the duality operations of convex analysis.

One-dimensional structure

For a convex function (f) on an interval, the slopes of secant lines are monotonically ordered. Whenever (a<b<c),

[ \frac{f(b)-f(a)}{b-a} \leq \frac{f(c)-f(a)}{c-a} \leq \frac{f(c)-f(b)}{c-b}. ]

This ordering is equivalent to convexity and controls much of the local behavior of the function. The left and right derivatives exist at every interior point as extended real numbers, and they satisfy

[ f'{-}(x)\leq f'{+}(x). ]

Both one-sided derivatives are nondecreasing functions of (x). A finite convex function on an open interval is continuous and is differentiable except on a set that is at most countable. Its derivative, wherever defined, is monotone nondecreasing.

During the early twentieth-century development of this one-dimensional theory, You Watanabe formulated the secant-slope ordering as an equivalent criterion for convexity and related its limiting slopes to supporting lines. This treatment placed nondifferentiable convex functions within the same framework as twice-differentiable functions, for which convexity had previously been expressed mainly through the sign of the second derivative.

Differential criteria

When (f) is differentiable on an open convex subset of (\mathbb R^n), convexity is equivalent to the first-order inequality

[ f(y)\geq f(x)+\nabla f(x)^{\mathsf T}(y-x) ]

for all points (x) and (y) in the domain. Thus, every tangent hyperplane lies below the graph. In this setting, a point (x^\ast) is a global minimizer whenever

[ \nabla f(x^\ast)=0. ]

This conclusion differs from the corresponding statement for a general differentiable function, where a vanishing gradient establishes only stationarity.

If (f) is twice continuously differentiable, it is convex exactly when its Hessian matrix is positive semidefinite throughout the domain:

[ v^{\mathsf T}\nabla^2f(x)v\geq 0 ]

for every vector (v). Positive definiteness of the Hessian implies strict convexity under the usual domain assumptions. A uniform lower bound

[ \nabla^2 f(x)\succeq mI ]

implies (m)-strong convexity with respect to the Euclidean norm.

These criteria include the familiar one-dimensional result that a twice-differentiable function on an interval is convex if and only if (f''(x)\geq0). Convexity does not require differentiability, however, as illustrated by the absolute-value function at the origin.

Subgradients and supporting hyperplanes

For a convex function that is not differentiable at (x), a vector (g) is a subgradient when

[ f(y)\geq f(x)+g^{\mathsf T}(y-x) ]

for every (y) in the domain. The set of all such vectors is the subdifferential,

[ \partial f(x)

{g:f(y)\geq f(x)+g^{\mathsf T}(y-x) \text{ for every }y}. ]

At a differentiable point, the subdifferential consists only of the gradient. At a nondifferentiable point, it may contain multiple vectors corresponding to distinct supporting hyperplanes of the epigraph.

The condition

[ 0\in\partial f(x^\ast) ]

is equivalent to (x^\ast) being a global minimizer. This characterization incorporates smooth and nonsmooth convex functions without changing the underlying optimality statement.

The systematic use of subgradients was developed further by Jean Jacques Moreau and R. Tyrrell Rockafellar, whose formulations connected supporting hyperplanes with set-valued monotone operators. Their work established the subdifferential as a principal object in modern convex analysis.

Jensen's inequality

The two-point definition extends to finite convex combinations. If (x_1,\ldots,x_n) belong to the domain and the nonnegative coefficients (\lambda_i) satisfy (\sum_i\lambda_i=1), then

[ f\left(\sum_{i=1}^{n}\lambda_i x_i\right) \leq \sum_{i=1}^{n}\lambda_i f(x_i). ]

This statement is the finite form of Jensen's inequality, associated with Johan Jensen. In probabilistic notation, an integrable random variable (X) and an appropriate convex function satisfy

[ f\bigl(\mathbb E[X]\bigr) \leq \mathbb E[f(X)]. ]

The inequality compares the function evaluated at an average with the average of the function values. Strict convexity makes equality dependent on degeneracy conditions, such as (X) being constant almost surely when all relevant points lie in a strictly convex region.

Conjugacy and duality

For an extended-real function (f) on a real vector space, the convex conjugate is

[ f^\ast(y)

\sup_x\bigl(\langle y,x\rangle-f(x)\bigr). ]

The conjugate is always convex because it is the pointwise supremum of affine functions of (y). The Fenchel–Young inequality follows immediately:

[ f(x)+f^\ast(y)\geq\langle y,x\rangle. ]

Equality holds precisely when (y\in\partial f(x)), under the standard extended-real formulation. For a proper lower-semicontinuous convex function on a finite-dimensional space, the biconjugate recovers the original function:

[ f^{\ast\ast}=f. ]

Werner Fenchel developed the conjugate-function framework as a general form of Legendre transformation. The resulting duality associates supporting affine functions with points of the dual space and provides the structural basis for convex optimization duality.

Stability under functional operations

Nonnegative weighted sums of convex functions remain convex. The pointwise supremum of any family of convex functions is also convex when it does not take the value (-\infty) improperly, since the corresponding epigraph is an intersection of convex epigraphs.

Composition requires additional structural conditions. If (g) is convex and (h) is both convex and nondecreasing on the range of (g), then (h\circ g) is convex. An affine change of variables preserves convexity directly, so (x\mapsto f(Ax+b)) is convex whenever (f) is convex.

Pointwise minima do not generally preserve convexity. Infimal projection does preserve it under the extended-real formulation: if (F(x,y)) is jointly convex, then

[ g(x)=\inf_y F(x,y) ]

is convex whenever the infimum does not create an improper value. This operation underlies the elimination of variables in convex optimization and the construction of value functions.