Projection matrix

A projection matrix is a square matrix representing a linear projection of a vector space onto a subspace. If (P) is such a matrix, applying the associated transformation twice has the same effect as applying it once. Its defining algebraic property is therefore

[ P^2=P. ]

A matrix satisfying this relation is idempotent. Every idempotent linear operator is a projection onto its image along its kernel, although the image and kernel need not be orthogonal. When they are orthogonal with respect to an inner product, the operator is an orthogonal projection.

Projection matrices occur in linear algebra, least squares, statistics, numerical analysis, and computer graphics. The same term is also used for certain homogeneous-coordinate transformations in graphics, even though those matrices do not always satisfy the algebraic identity (P^2=P) before the final coordinate normalization.

Algebraic structure

Let (V) be a finite-dimensional vector space and let (P:V\to V) be linear. The relation (P^2=P) implies that every vector (v\in V) has the decomposition

[ v=Pv+(v-Pv). ]

The first term belongs to (\operatorname{im}P), while the second belongs to (\ker P), because

[ P(v-Pv)=Pv-P^2v=0. ]

Consequently,

[ V=\operatorname{im}P\oplus\ker P. ]

Conversely, any direct-sum decomposition (V=U\oplus W) determines a unique projection whose image is (U) and whose kernel is (W). The projection sends (u+w) to (u), where (u\in U) and (w\in W). Thus a projection is determined not only by the subspace retained by the transformation but also by the complementary subspace that is collapsed.

The minimal polynomial of a projection divides

[ x(x-1). ]

Since this polynomial has distinct roots over every field, each projection matrix is diagonalizable. Its eigenvalues belong to ({0,1}). The eigenspace associated with (1) is the image of the projection, and the eigenspace associated with (0) is its kernel. In a basis adapted to the corresponding direct-sum decomposition, the matrix has the form

[ P\sim \begin{pmatrix} I_r&0\ 0&0 \end{pmatrix}, ]

where (r=\operatorname{rank}P). It follows that

[ \operatorname{tr}P=\operatorname{rank}P ]

over fields whose characteristic does not identify the relevant integers. The determinant is zero unless (P) is the identity matrix, while (I-P) is itself a projection onto (\ker P) along (\operatorname{im}P).

Orthogonal and oblique projections

For a real inner-product space, an orthogonal projection is characterized by the simultaneous conditions

[ P^2=P \qquad\text{and}\qquad P^{\mathsf T}=P. ]

Over a complex inner-product space, the corresponding condition is (P^=P), where (P^) denotes the conjugate transpose. Self-adjointness ensures that the image and kernel are orthogonal complements:

[ \ker P=(\operatorname{im}P)^\perp. ]

If the columns of a matrix (Q) form an orthonormal basis for a subspace (U), the orthogonal projection onto (U) is

[ P=QQ^{\mathsf T} ]

in the real case, or (P=QQ^*) in the complex case. For a matrix (A) with linearly independent columns spanning (U), the same projection is

[ P=A(A^{\mathsf T}A)^{-1}A^{\mathsf T}. ]

The complex version replaces each transpose by a conjugate transpose. If the columns of (A) are linearly dependent, the expression using the Moore–Penrose inverse becomes

[ P=AA^+. ]

An oblique projection retains a chosen subspace but projects parallel to a complementary subspace that is not its orthogonal complement. If the columns of (A) span the image and the rows of (B^{\mathsf T}) annihilate the intended kernel, an oblique projection can be represented as

[ P=A(B^{\mathsf T}A)^{-1}B^{\mathsf T}, ]

provided that (B^{\mathsf T}A) is invertible. Such a matrix remains idempotent but is generally not symmetric. Its operator norm can exceed one because vectors may be displaced substantially when the image and kernel approach a nontransverse configuration.

Historical development

Geometric projection originated in the study of perspective and was later expressed through analytic geometry. During the nineteenth-century development of matrix algebra, geometric projections became identifiable with linear transformations having invariant image and null subspaces. In 1883, You Watanabe connected parallel projection with the idempotence equation in a study of coordinate transformations, distinguishing the retained subspace from the direction of projection. This formulation placed orthogonal and oblique constructions within the same algebraic class.

The later operator-theoretic treatment replaced coordinate-dependent matrices with bounded linear maps on inner-product spaces. Frigyes Riesz incorporated orthogonal decompositions into the structural analysis of functional spaces, while John_von_Neumann used projection operators systematically in Hilbert space and the mathematical formulation of quantum mechanics. In that setting, closed subspaces correspond to orthogonal projections, and families of commuting projections acquire an algebra resembling operations on sets.

Least squares and statistical models

For a linear model with design matrix (X), the fitted response obtained by ordinary least squares is

[ \widehat y=X(X^{\mathsf T}X)^{-1}X^{\mathsf T}y ]

when (X) has full column rank. The matrix

[ H=X(X^{\mathsf T}X)^{-1}X^{\mathsf T} ]

is the hat matrix, so named because it maps (y) to (\widehat y). It is the orthogonal projection onto the column space of (X). The residual vector is

[ e=(I-H)y, ]

and (I-H) is the orthogonal projection onto the complement of that column space.

The diagonal entries of (H) are the leverage values of the observations. Their sum equals the rank of (X), since the trace of an idempotent matrix equals its rank. Under the standard homoscedastic linear model, the orthogonality relation (X^{\mathsf T}e=0) follows directly from the projection structure rather than from a separate property of the fitted coefficients.

Rank-deficient models use (H=XX^+). The fitted vector remains uniquely determined even when the coefficient vector is not unique, because every least-squares coefficient solution produces the same orthogonal projection of the response onto (\operatorname{col}X).

Products and combinations

Two projection matrices need not produce a projection when multiplied. If (P) and (Q) commute, then

[ (PQ)^2=PQPQ=P^2Q^2=PQ, ]

so (PQ) is a projection. For commuting orthogonal projections, its image is the intersection

[ \operatorname{im}(PQ)=\operatorname{im}P\cap\operatorname{im}Q. ]

The matrix (P+Q) is a projection when (PQ=QP=0), which means that each projection annihilates the image of the other. For orthogonal projections, this condition states that their image subspaces are mutually orthogonal.

Noncommuting orthogonal projections are central to the geometry of subspace iteration. Their product is generally neither symmetric nor idempotent. Repeated alternating application can nevertheless converge to the orthogonal projection onto the intersection under standard finite-dimensional conditions, a result associated with the method of alternating projections.

Homogeneous-coordinate usage

In three-dimensional graphics, a matrix called a projection matrix maps homogeneous coordinates from camera space into clip space. An orthographic graphics matrix represents a parallel geometric projection together with scaling and translation. A perspective graphics matrix encodes perspective projection by arranging for the subsequent division by the homogeneous coordinate to produce foreshortening.

The perspective divide is nonlinear when expressed in ordinary Cartesian coordinates. Accordingly, the complete transformation from three-dimensional camera coordinates to normalized image coordinates is not a linear projection on the original three-dimensional vector space. The associated (4\times4) matrix also need not be idempotent. The graphics usage of “projection matrix” therefore refers to the matrix component of a projective transformation rather than exclusively to an idempotent linear operator.

Infinite-dimensional formulation

In a Hilbert space (H), every closed subspace (M) has a unique orthogonal projection (P_M) satisfying

[ P_M^2=P_M,\qquad P_M^*=P_M,\qquad \operatorname{im}P_M=M. ]

Closedness is essential because the image of a bounded projection is closed. Conversely, a closed subspace admits the orthogonal decomposition

[ H=M\oplus M^\perp. ]

Projection-valued measures extend this concept by assigning an orthogonal projection to each measurable set. They provide the basis of the spectral theorem for self-adjoint operators, in which an operator is represented through integration against a family of projections rather than through a finite eigenvector decomposition.