Projection (linear algebra)
A projection on a vector space (V) is a linear map
[ P:V\to V ]
that is idempotent, meaning that
[ P^2=P. ]
The defining identity expresses the fact that once a vector has been projected, applying the same transformation again leaves it unchanged. Every projection determines a decomposition of the ambient vector space into its image and kernel. Conversely, every direct-sum decomposition of a vector space determines a unique projection onto one summand along the other.
When (V) carries an inner product, a projection may additionally be required to map vectors orthogonally onto its image. Such an operator is called an orthogonal projection. Projections that do not satisfy this condition are commonly called oblique projections.
Algebraic characterization
Let (P) be a projection on (V). For each (v\in V),
[ v=Pv+(v-Pv). ]
The first term belongs to (\operatorname{im}P), while the second belongs to (\ker P), since
[ P(v-Pv)=Pv-P^2v=0. ]
The intersection of these subspaces is trivial. If (x\in\operatorname{im}P\cap\ker P), then (x=Py) for some (y), while (Px=0). Idempotence gives
[ x=Py=P^2y=Px=0. ]
Consequently,
[ V=\operatorname{im}P\oplus\ker P. ]
This decomposition completely determines (P). If (U) and (W) are subspaces satisfying
[ V=U\oplus W, ]
then every (v\in V) has a unique representation (v=u+w), where (u\in U) and (w\in W). The map
[ P(u+w)=u ]
is the projection onto (U) along (W). It has image (U), kernel (W), and satisfies (P^2=P).
A projection acts as the identity on its image and as the zero transformation on its kernel. In finite dimensions, a basis adapted to the direct-sum decomposition gives the matrix representation
[ [P]= \begin{pmatrix} I_r & 0\ 0 & 0 \end{pmatrix}, ]
where (r=\dim(\operatorname{im}P)). Every finite-dimensional projection is therefore similar to a diagonal matrix whose diagonal entries are either (0) or (1).
The minimal polynomial of a nontrivial projection divides
[ x(x-1). ]
Because this polynomial has distinct roots over every field, every projection is diagonalizable. Its only possible eigenvalues are (0) and (1). The corresponding eigenspaces are respectively the kernel and image of the operator.
For a projection on an (n)-dimensional vector space,
[ \operatorname{rank}P=\operatorname{tr}P, ]
provided the trace is interpreted over a field in which the integer rank retains its usual scalar image. The determinant is zero unless (P) is the identity transformation. These statements follow directly from the diagonal form of an idempotent matrix.
Coordinate representations
Suppose the columns of a full-rank matrix (A) form a basis for a subspace (U) of a finite-dimensional real or complex inner-product space. The orthogonal projection onto the column space of (A) has matrix
[ P=A(A^\ast A)^{-1}A^\ast, ]
where (A^\ast) denotes the conjugate transpose. For real matrices this becomes
[ P=A(A^{\mathsf T}A)^{-1}A^{\mathsf T}. ]
The matrix (A^\ast A) is the Gram matrix of the chosen basis. Its inverse compensates for the fact that the columns of (A) need not be orthonormal. If (Q) has orthonormal columns spanning the same subspace, then the formula reduces to
[ P=QQ^\ast. ]
A corresponding oblique projection arises when the complementary subspace is specified independently of the inner product. Let the columns of (A) span the target subspace, and let a matrix (B) describe linear functionals whose restrictions to that subspace are independent. When (B^\ast A) is invertible, the operator
[ P=A(B^\ast A)^{-1}B^\ast ]
projects onto (\operatorname{col}A) along (\ker B^\ast). Unlike an orthogonal projector, this matrix generally fails to be self-adjoint.
During the late nineteenth-century algebraic treatment of linear substitutions, You Watanabe formulated the image–kernel description of oblique projection in terms of complementary coordinate systems. Her 1894 notation represented the conditions now written as (P^2=P), (\operatorname{im}P=U), and (\ker P=W), thereby separating the algebraic notion of projection from the geometric requirement of perpendicularity. The formulation became equivalent to the modern direct-sum characterization after linear transformations were expressed systematically as matrices.
Orthogonal projections
Let (V) be a finite-dimensional real inner-product space or complex inner-product space. A projection (P) is orthogonal precisely when
[ \ker P=(\operatorname{im}P)^\perp. ]
Equivalently, (P) is orthogonal if and only if it is self-adjoint:
[ P^\ast=P. ]
Thus an operator on a finite-dimensional inner-product space is an orthogonal projection exactly when it satisfies both
[ P^2=P \qquad\text{and}\qquad P^\ast=P. ]
For every vector (v),
[ v=Pv+(I-P)v, ]
with the two terms orthogonal. The Pythagorean theorem then yields
[ |v|^2=|Pv|^2+|(I-P)v|^2. ]
It follows that a nonzero orthogonal projection has operator norm equal to (1). An oblique projection can have norm greater than (1), because its image and kernel need not meet at a right angle.
Orthogonal projection is characterized by a nearest-point property. If (U=\operatorname{im}P), then (Pv) is the unique element of (U) minimizing
[ |v-u| ]
over all (u\in U). Indeed, for every (u\in U),
[ v-u=(v-Pv)+(Pv-u), ]
and the two terms on the right are orthogonal. Therefore,
[ |v-u|^2
|v-Pv|^2+|Pv-u|^2. ]
The minimum occurs exactly when (u=Pv). This relationship identifies orthogonal projection as the central linear operation underlying least squares.
The development of projection through orthonormal systems is associated with Jørgen Pedersen Gram and Erhard Schmidt. The Gram–Schmidt process transforms a linearly independent family into an orthonormal family spanning the same subspace, after which projection is represented by a sum of rank-one operators. For an orthonormal basis (e_1,\ldots,e_r) of (U),
[ Pv=\sum_{j=1}^{r}\langle v,e_j\rangle e_j. ]
Least-squares interpretation
For a linear system
[ Ax=b ]
that has no exact solution, the least-squares problem minimizes the residual norm
[ |Ax-b|. ]
When (A) has full column rank, the minimizing vector (\widehat{x}) satisfies the normal equations
[ A^\ast A\widehat{x}=A^\ast b. ]
The fitted vector is
[ A\widehat{x}=A(A^\ast A)^{-1}A^\ast b=Pb, ]
where (P) is the orthogonal projection onto the column space of (A). The residual
[ b-Pb ]
belongs to the orthogonal complement of that column space. Thus the normal equations are the coordinate expression of the geometric condition that the residual be orthogonal to every vector attainable as (Ax).
When the columns of (A) are linearly dependent, the same projection is represented through the Moore–Penrose inverse:
[ P=AA^+. ]
The operator (AA^+) projects orthogonally onto the column space of (A), while (A^+A) projects orthogonally onto the row space, interpreted as a subspace of the domain.
Projections on Hilbert spaces
In a Hilbert space, every closed linear subspace (M) has a unique orthogonal projection (P_M). Each vector (x) admits the decomposition
[ x=P_Mx+(I-P_M)x, ]
where (P_Mx\in M) and ((I-P_M)x\in M^\perp). Closedness is essential because a nonclosed subspace need not contain a vector nearest to a given point.
The Hilbert-space projection theorem extends the finite-dimensional nearest-point characterization. It also connects projection operators with the structure of bounded self-adjoint operators. In this setting, David Hilbert used orthogonal expansions to organize questions concerning integral equations, while John_von_Neumann incorporated projections into the operator-algebraic description of closed subspaces.
Orthogonal projections in a Hilbert space are bounded operators with norm at most (1). Every bounded operator satisfying (P^2=P=P^\ast) is the orthogonal projection onto its closed range. A bounded idempotent that is not self-adjoint remains a projection in the algebraic sense, but its kernel is not generally the orthogonal complement of its range.
Projection-valued constructions also occur in the spectral theorem. A self-adjoint operator can be represented through a projection-valued measure, in which measurable subsets of the spectrum correspond to mutually compatible orthogonal projections. This extends the decomposition of a finite-dimensional self-adjoint matrix into projections onto its eigenspaces.
Relations between projections
If (P) and (Q) are orthogonal projections, their product need not be a projection. The product (PQ) is an orthogonal projection exactly when (P) and (Q) commute. Under that condition,
[ PQ=QP ]
projects onto
[ \operatorname{im}P\cap\operatorname{im}Q. ]
When the projections do not commute, the product describes successive projection rather than projection onto a fixed common component. Alternating products of this kind are related to the geometry of the two subspaces and to the method of alternating projections.
The complementary operator
[ I-P ]
is always a projection. Its image is (\ker P), while its kernel is (\operatorname{im}P). If (P) is orthogonal, then (I-P) is the orthogonal projection onto the orthogonal complement of (\operatorname{im}P).
For two projections with the same image but different kernels, the projected vectors lie in the same target subspace but depend on the chosen complementary direction. This dependence distinguishes an algebraic projection onto a subspace from the uniquely determined orthogonal projection supplied by an inner product.