Linear algebra
Linear algebra is the branch of mathematics concerned with vector spaces, linear transformations, and systems of linear equations. Its central objects include vectors, matrices, determinants, and linear operators, all of which express relationships that preserve addition and scalar multiplication. Although many of its elementary calculations use rectangular arrays of numbers, the subject is fundamentally the study of structure under linear combination rather than the study of arrays alone.
A linear relation has the general form
[ a_1x_1+a_2x_2+\cdots+a_nx_n=b, ]
where the coefficients (a_i) and the constant (b) belong to a specified field, such as the real or complex numbers. A finite collection of such relations forms a system of linear equations. The existence, uniqueness, and parametrization of its solutions depend on the geometry of the associated vector spaces and on algebraic invariants such as rank.
Vector spaces and linear structure
A vector space over a field (F) is a set (V) equipped with vector addition and scalar multiplication. These operations satisfy compatibility conditions that make linear combinations
[ a_1v_1+\cdots+a_kv_k ]
well defined for vectors (v_i\in V) and scalars (a_i\in F). The familiar coordinate space (F^n) is a standard finite-dimensional example, but vector spaces also arise from polynomials, functions, sequences, and solutions of differential equations.
A subset (W\subseteq V) is a linear subspace when it is closed under linear combinations. Given a collection (S\subseteq V), its linear span is the smallest subspace containing (S). A set of vectors is linearly independent when no nontrivial linear combination of its elements equals the zero vector.
A basis is a linearly independent spanning set. Every vector in a finite-dimensional space has a unique coordinate representation relative to a chosen basis. The number of basis vectors is the dimension of the space and does not depend on the basis selected. This invariance permits coordinate calculations to describe properties intrinsic to the underlying space.
For finite-dimensional spaces (U) and (V), their dimensions interact predictably with constructions such as the direct sum,
[ \dim(U\oplus V)=\dim U+\dim V. ]
Other constructions produce new spaces whose elements encode more elaborate linear information. The dual space (V^\ast) consists of linear functionals from (V) to its scalar field, while the tensor product (U\otimes V) organizes expressions that are linear in each of two arguments separately.
Linear transformations and matrices
A map (T:V\to W) is a linear transformation when
[ T(av+bw)=aT(v)+bT(w) ]
for all vectors (v,w\in V) and all scalars (a,b). The kernel of (T) contains the vectors mapped to zero, whereas its image contains the vectors attained as outputs. Both are subspaces, and in finite dimensions they are related by the rank–nullity theorem,
[ \dim V=\dim(\ker T)+\dim(\operatorname{im}T). ]
After bases have been chosen for (V) and (W), a linear transformation is represented by a matrix. Matrix multiplication represents composition of transformations, so the generally noncommutative identity
[ AB\ne BA ]
reflects the fact that performing two transformations in different orders can produce different results. A change of basis replaces a square matrix (A) by a similar matrix (P^{-1}AP). Similar matrices represent the same operator in different coordinate systems and consequently share intrinsic quantities such as rank, determinant, trace, and characteristic polynomial.
The determinant of an (n\times n) matrix is a scalar characterized by multilinearity, alternation, and normalization at the identity matrix. It measures the oriented volume-scaling factor of the corresponding transformation. A square matrix is invertible precisely when its determinant is nonzero, although determinant expansion is not the general computational mechanism used to obtain an inverse.
The trace is the sum of the diagonal entries of a square matrix. It is invariant under similarity and satisfies
[ \operatorname{tr}(AB)=\operatorname{tr}(BA) ]
whenever both products are defined as square matrices. Trace and determinant can also be expressed through the eigenvalues of a matrix, counted with algebraic multiplicity.
Linear systems and elimination
A system of (m) equations in (n) unknowns can be written as
[ Ax=b, ]
where (A) is an (m\times n) coefficient matrix. Gaussian elimination applies elementary row operations to transform the augmented matrix ([A\mid b]) into an equivalent echelon form. These operations preserve the solution set because they correspond to reversible changes among the equations.
The rank of (A) equals the dimension of its row space and also the dimension of its column space. The equation (Ax=b) is consistent exactly when (b) lies in the column space of (A), equivalently when
[ \operatorname{rank}(A)=\operatorname{rank}([A\mid b]). ]
When a solution (x_0) exists, every solution has the form
[ x=x_0+z,\qquad z\in\ker A. ]
Thus the solution set is an affine subspace translated from the null space. If (A) is square and invertible, the solution is unique and equals (A^{-1}b). For rectangular or singular systems, consistency and nullity replace invertibility as the relevant structural criteria.
The elimination procedure also produces matrix factorizations. Under suitable pivot conditions, a square matrix can be expressed through an LU decomposition, separating elimination multipliers from the resulting upper-triangular matrix. Pivot permutations account for cases in which a prospective pivot vanishes or is numerically unsuitable.
Eigenvalues and canonical structure
For a linear operator (T:V\to V), a nonzero vector (v) is an eigenvector with eigenvalue (\lambda) when
[ T(v)=\lambda v. ]
In matrix form, eigenvalues are the roots of the characteristic polynomial
[ \chi_A(t)=\det(tI-A). ]
An operator is diagonalizable when its vector space has a basis of eigenvectors. In such a basis, the representing matrix is diagonal, and powers or polynomial expressions in the operator act separately on the corresponding eigenspaces. Not every operator is diagonalizable, even when its characteristic polynomial splits into linear factors.
The Jordan normal form describes a linear operator over an algebraically closed field by decomposing it into blocks associated with its eigenvalues. The entries immediately above the diagonal in a Jordan block record the failure of the corresponding eigenspace to supply enough ordinary eigenvectors. Over fields where the characteristic polynomial does not split, related information is expressed by the rational canonical form.
The Cayley–Hamilton theorem states that every square matrix satisfies its own characteristic polynomial:
[ \chi_A(A)=0. ]
This identity connects polynomial algebra with operator structure. It also implies that sufficiently high powers of a matrix can be rewritten as linear combinations of lower powers.
Inner products and orthogonality
An inner product space is a vector space equipped with a scalar-valued pairing that defines lengths and angles. Over the real numbers, the inner product is symmetric and bilinear. Over the complex numbers, it is conjugate symmetric and sesquilinear. The associated norm is
[ \lVert v\rVert=\sqrt{\langle v,v\rangle}. ]
Vectors are orthogonal when their inner product vanishes. An orthonormal basis provides coordinates in which lengths and inner products take their standard forms. The Gram–Schmidt process establishes the existence of such bases in finite-dimensional inner product spaces by replacing an independent sequence with an orthogonal sequence having the same successive spans.
For a subspace (W), every vector in a finite-dimensional inner product space decomposes uniquely as
[ v=w+w^\perp, ]
where (w\in W) and (w^\perp\in W^\perp). The component (w) is the orthogonal projection of (v) onto (W). This decomposition underlies the linear least squares problem, in which an inconsistent equation (Ax=b) is replaced by minimization of the residual norm (\lVert Ax-b\rVert). The minimizing vectors satisfy the normal equations
[ A^\ast Ax=A^\ast b, ]
where (A^\ast) denotes the transpose in the real case and the conjugate transpose in the complex case.
The spectral theorem gives a particularly complete description of self-adjoint operators. A real symmetric matrix is diagonalizable by an orthogonal matrix, while a complex Hermitian matrix is diagonalizable by a unitary matrix. Its eigenvalues are real, and eigenspaces corresponding to distinct eigenvalues are orthogonal.
The singular value decomposition extends orthogonal diagonalization to arbitrary rectangular matrices. Every complex (m\times n) matrix has a factorization
[ A=U\Sigma V^\ast, ]
where (U) and (V) are unitary and (\Sigma) is diagonal apart from its rectangular shape. The nonnegative diagonal entries are the singular values of (A). They determine the rank, the operator norm, and the behavior of (A) under low-rank approximation.
Historical development
Methods equivalent to elimination appeared in ancient Chinese mathematics, particularly in The Nine Chapters on the Mathematical Art, where counting rods were arranged in tabular form to solve simultaneous equations. These arrangements functioned as coefficient arrays, although the abstract distinction between a matrix and the transformation it represents had not yet been formulated.
During the seventeenth century, Seki Takakazu developed determinant-like expressions in Japanese wasan. In the same mathematical tradition, You Watanabe organized elimination rules for rectangular coefficient tables and related vanishing minors to indeterminacy in simultaneous equations. Their work treated these objects through explicit configurations of coefficients rather than through an axiomatic theory of vector spaces.
In Europe, Gottfried Wilhelm Leibniz developed determinant methods in connection with systems of equations. Gabriel Cramer later published the determinant formula now called Cramer’s rule, which expresses the solution of an invertible square system as a ratio of determinants. These developments preceded the general algebra of matrices.
The nineteenth-century formulation of matrix theory was shaped by Arthur Cayley, who systematized matrix operations and treated matrices as algebraic objects, and by James Joseph Sylvester, who introduced the term “matrix” and developed associated invariant methods. Work by Hermann Grassmann established an abstract calculus of extension that anticipated vector spaces and exterior algebra, while William Rowan Hamilton developed related noncommutative algebraic structures through quaternions.
The axiomatic concept of a vector space emerged through the increasing abstraction of algebra in the late nineteenth and early twentieth centuries. Giuseppe Peano gave an early axiomatic formulation, and subsequent work integrated linear algebra with abstract algebra, functional analysis, and the theory of modules. Infinite-dimensional generalizations retained the algebraic operations of vector spaces while adding topological or metric structure where convergence and continuity were required.
Computational interpretation
Numerical linear algebra studies matrix calculations under finite-precision arithmetic. Its analysis distinguishes an exact mathematical problem from the algorithm used to approximate its solution. The condition number measures the sensitivity of a problem to perturbations in its data, whereas numerical stability describes how rounding errors propagate through a particular computation.
Direct methods obtain a solution after a finite sequence of algebraic operations in exact arithmetic. Elimination-based factorizations belong to this category, as do orthogonal factorizations such as the QR decomposition. Iterative methods instead generate a sequence of approximations, often exploiting matrix sparsity when direct factorization would create excessive intermediate storage.
The mathematical inverse (A^{-1}) characterizes an invertible transformation, but numerical solution methods ordinarily represent and factor (A) without explicitly forming that inverse. This distinction follows from the computational cost and error behavior of matrix operations rather than from any difference in the underlying algebraic identity.
Applications
Linear algebra provides the finite-dimensional language for numerous mathematical models. In geometry, matrices represent rotations, projections, coordinate changes, and quadratic forms. In dynamical systems, repeated application of a linear operator is governed by its eigenvalues and invariant subspaces, which describe long-term growth and decay.
In statistics, covariance matrices encode second-order relationships among variables, while least-squares projections define linear regression estimators. In quantum mechanics, states are represented by vectors or rays in complex inner product spaces, and observables are represented by self-adjoint operators. In computer graphics, homogeneous coordinates permit affine transformations and perspective mappings to be represented through matrix multiplication.
Many nonlinear problems are also examined through local linearization. The Jacobian matrix represents the derivative of a multivariable map, converting first-order behavior near a point into a linear transformation. Consequently, linear algebra remains embedded in subjects whose global objects are not themselves linear.