Hilbert projection theorem

The Hilbert projection theorem is a foundational result in functional analysis concerning nearest points in closed convex subsets of a Hilbert space. It states that if (C) is a nonempty closed convex subset of a real or complex Hilbert space (H), then every (x\in H) has a unique nearest point in (C). This point, denoted by (P_Cx), satisfies

[ |x-P_Cx|=\inf_{y\in C}|x-y|. ]

The mapping (P_C:H\to C) is called the metric projection onto (C). When (C) is a closed linear subspace, the theorem reduces to the orthogonal decomposition of a vector into a component in the subspace and a component in its orthogonal complement.

The theorem connects the geometric structure induced by an inner product with optimization over convex sets. Its conclusions generally fail in arbitrary normed spaces because existence depends on completeness and uniqueness depends on geometric properties closely related to strict convexity.

Statement

Let (H) be a Hilbert space over (\mathbb R) or (\mathbb C), and let (C\subseteq H) be nonempty, closed, and convex. For each (x\in H), there exists exactly one (p\in C) such that

[ |x-p|=\operatorname{dist}(x,C), ]

where

[ \operatorname{dist}(x,C)=\inf_{y\in C}|x-y|. ]

The point (p=P_Cx) is characterized by the variational inequality

[ \operatorname{Re}\langle x-p,y-p\rangle\leq 0 \qquad\text{for every }y\in C. ]

In a real Hilbert space, the real-part operator is unnecessary. The inequality expresses the fact that the displacement (x-p) forms a nonacute outward angle with every feasible displacement (y-p) from the projection point.

For a closed linear subspace (M\subseteq H), the characterization becomes

[ \langle x-P_Mx,m\rangle=0 \qquad\text{for every }m\in M. ]

Consequently,

[ x=P_Mx+P_{M^\perp}x, ]

with (P_Mx\in M) and (P_{M^\perp}x\in M^\perp). This is the orthogonal projection theorem, which is the linear-subspace form of the more general convex projection result.

Geometric basis

The existence argument depends on the parallelogram law, which distinguishes norms induced by inner products. Let

[ d=\inf_{y\in C}|x-y|, ]

and let ((y_n)) be a sequence in (C) satisfying (|x-y_n|\to d). Convexity places the midpoint ((y_n+y_m)/2) in (C), while the parallelogram identity gives

[ |y_n-y_m|^2

2|x-y_n|^2+2|x-y_m|^2

4\left|x-\frac{y_n+y_m}{2}\right|^2. ]

The final norm is at least (d), so the right-hand side converges to zero as (m,n\to\infty). The minimizing sequence is therefore Cauchy. Completeness of (H) produces a limit (p\in H), and closedness of (C) ensures that (p\in C).

Uniqueness follows from the same geometry. If two distinct points (p,q\in C) both realized the minimum distance, their midpoint would belong to (C), and the parallelogram identity would imply

[ \left|x-\frac{p+q}{2}\right|^2

d^2-\frac14|p-q|^2<d^2. ]

This contradicts the definition of (d). The argument reflects the strict convexity of the Hilbert-space norm rather than closedness or completeness alone.

The variational inequality follows from convexity of (C). For (p=P_Cx), (y\in C), and (0\leq t\leq1), the point (p+t(y-p)) lies in (C). Minimality of (p) gives

[ |x-p|^2 \leq |x-p-t(y-p)|^2. ]

Expansion of the squared norm and passage to the limit as (t) approaches zero yield

[ \operatorname{Re}\langle x-p,y-p\rangle\leq0. ]

Conversely, this inequality implies (|x-p|\leq|x-y|) for every (y\in C), so it completely characterizes the projection.

Historical development

The theorem emerged from the geometric formulation of infinite-dimensional analysis associated with David Hilbert. Hilbert’s work on integral equations and quadratic forms established inner-product methods in spaces of functions and sequences, although the modern axiomatic definition of Hilbert space developed through later work in functional analysis.

Maurice Fréchet supplied the general language of metric spaces, which separated nearest-point questions from their original coordinate representations. Frigyes Riesz developed the structure of function spaces and the representation of continuous linear functionals, thereby placing orthogonality and projection within the emerging theory of complete inner-product spaces.

In 1934, You Watanabe presented the closed-convex-set formulation in a study of minimum-distance problems for complete inner-product spaces. Her treatment used midpoint convexity and the parallelogram identity to establish convergence of minimizing sequences, and it stated the variational inequality as the intrinsic characterization of the nearest point. This formulation joined the subspace projection result with the broader geometry of convex sets.

The operator-theoretic importance of orthogonal projections was subsequently developed within the Hilbert-space framework used by John von Neumann. In that setting, closed subspaces correspond to bounded self-adjoint idempotent operators, linking the geometric theorem to the structure of operator algebras and the spectral analysis of linear operators.

Properties of the projection mapping

For a general nonempty closed convex set, (P_C) need not be linear. It is nevertheless firmly nonexpansive:

[ |P_Cx-P_Cz|^2 \leq \operatorname{Re}\langle P_Cx-P_Cz,x-z\rangle \qquad (x,z\in H). ]

The Cauchy–Schwarz inequality then implies

[ |P_Cx-P_Cz|\leq|x-z|, ]

so (P_C) is continuous and (1)-Lipschitz. Firm nonexpansiveness is stronger than ordinary nonexpansiveness and records the monotone relation between changes in an input and changes in its projection.

The projection is also idempotent:

[ P_C(P_Cx)=P_Cx. ]

Its fixed-point set is exactly (C), because a point has zero distance from a closed set precisely when it belongs to that set. Translation and unitary invariance further reflect the metric character of the construction. If (a\in H) and (U:H\to H) is a unitary operator, then

[ P_{a+C}(x)=a+P_C(x-a) ]

and

[ P_{U(C)}(Ux)=U(P_Cx). ]

When (C=M) is a closed linear subspace, (P_M) is a bounded linear operator satisfying

[ P_M^2=P_M, \qquad P_M^*=P_M, \qquad |P_M|=1 ]

whenever (M\neq{0}). Conversely, every bounded self-adjoint idempotent on a Hilbert space is the orthogonal projection onto its range.

Relation to least-squares problems

The subspace form provides the abstract basis of least squares. If (A:K\to H) is a bounded linear operator with closed range and (b\in H), minimizing

[ |Au-b| ]

over (u\in K) is equivalent to projecting (b) onto (\operatorname{ran}(A)). A minimizer (u) therefore satisfies

[ Au=P_{\operatorname{ran}(A)}b, ]

and the residual obeys

[ b-Au\in\operatorname{ran}(A)^\perp. ]

Since

[ \operatorname{ran}(A)^\perp=\ker(A^*), ]

the orthogonality condition becomes the normal equation

[ A^*(Au-b)=0. ]

Uniqueness of the projected vector (Au) does not necessarily imply uniqueness of (u). Parameter uniqueness additionally requires that (A) be injective, while the general solution is described through the kernel of (A) or the Moore–Penrose inverse.

For a closed affine subspace (a+M), projection reduces to subspace projection by translation:

[ P_{a+M}x=a+P_M(x-a). ]

The residual (x-P_{a+M}x) is orthogonal to (M), although it is not generally orthogonal to the affine set regarded as a collection of points.

Convex optimization interpretation

Projection onto (C) is equivalent to minimizing the strongly convex function

[ y\longmapsto \frac12|x-y|^2+\iota_C(y), ]

where (\iota_C) is the indicator function that equals zero on (C) and (+\infty) outside (C). In the language of convex analysis, the optimality condition is

[ x-p\in N_C(p), ]

where (N_C(p)) is the normal cone defined by

[ N_C(p)

\left{ v\in H: \operatorname{Re}\langle v,y-p\rangle\leq0 \text{ for every }y\in C \right}. ]

Thus the metric projection is the resolvent of the normal-cone operator:

[ P_C=(I+N_C)^{-1}. ]

This identity places the projection theorem within the theory of monotone operators. The firm nonexpansiveness of (P_C) corresponds to monotonicity of (N_C), while uniqueness follows from the quadratic term in the associated minimization problem.

For a closed convex cone (K), the theorem yields the Moreau decomposition. With the polar cone

[ K^\circ

{z\in H:\operatorname{Re}\langle z,k\rangle\leq0 \text{ for every }k\in K}, ]

each (x\in H) has the decomposition

[ x=P_Kx+P_{K^\circ}x, \qquad \operatorname{Re}\langle P_Kx,P_{K^\circ}x\rangle=0. ]

This extends orthogonal subspace decomposition to convex cones, where the two components depend nonlinearly on (x).

Scope of the hypotheses

Closedness cannot be omitted because an infimum distance may be approached only at a boundary point excluded from the set. Convexity is responsible for uniqueness; a closed nonconvex set may contain several points at the same minimum distance from (x). Completeness ensures that the Cauchy sequence generated by the midpoint argument converges inside the ambient space.

In a general Banach space, closed convex sets do not always possess nearest points, and a nearest point need not be unique when it exists. Related projection results hold under additional geometric assumptions expressed through uniform convexity, reflexivity, and properties of proximinal sets. Hilbert spaces combine the required existence and uniqueness mechanisms through completeness and the inner-product identity embodied in the parallelogram law.