Rayleigh quotient

The Rayleigh quotient is a scalar-valued function associated with a nonzero vector and a square matrix. For a matrix (A\in\mathbb{C}^{n\times n}) and a vector (x\in\mathbb{C}^n\setminus{0}), it is defined by

[ \rho_A(x)=\frac{x^{}Ax}{x^{}x}, ]

where (x^{*}) denotes the conjugate transpose of (x). When (A) and (x) are real, the definition reduces to

[ \rho_A(x)=\frac{x^{\mathsf T}Ax}{x^{\mathsf T}x}. ]

The quotient is invariant under multiplication of (x) by a nonzero scalar. It therefore depends only on the one-dimensional subspace spanned by (x), rather than on a particular normalization of that vector. This scale invariance connects the quotient with projective space, constrained optimization, and the geometry of eigenvalue problems.

For a Hermitian matrix, the Rayleigh quotient is real and provides a variational description of the spectrum. Its stationary values are eigenvalues, while its stationary directions are the corresponding eigenvectors. These properties make the quotient a central object in spectral theory, the analysis of differential operators, and numerical methods for approximating eigenpairs.

Historical development

The quotient takes its name from John William Strutt, 3rd Baron Rayleigh, who used ratios of quadratic expressions in his study of vibration and acoustics. In mechanical form, the numerator represented an energy determined by stiffness, while the denominator represented an energy determined by mass or inertia. The ratio supplied an approximation to the square of a natural frequency when evaluated on an approximate mode shape.

Walther Ritz subsequently developed a finite-dimensional variational treatment in which trial functions reduced boundary-value problems to matrix eigenvalue problems. The resulting Rayleigh–Ritz method interprets spectral approximation as optimization over a selected subspace. Richard Courant incorporated the same structure into the min–max characterization of ordered eigenvalues, thereby placing the quotient within the general theory of self-adjoint operators.

The terminology “quotient” refers to the ratio of two quadratic forms. It does not imply componentwise division, and it has no mathematical connection with an individual ray in Euclidean geometry, despite the projective interpretation of its scale invariance.

Spectral characterization

Suppose that (A) is Hermitian, with ordered eigenvalues

[ \lambda_1\leq \lambda_2\leq\cdots\leq\lambda_n. ]

The spectral theorem gives an orthonormal eigenbasis (q_1,\ldots,q_n). Writing

[ x=\sum_{i=1}^{n}c_iq_i ]

produces the identity

[ \rho_A(x)

\frac{\sum_{i=1}^{n}\lambda_i|c_i|^2} {\sum_{i=1}^{n}|c_i|^2}. ]

The quotient is consequently a weighted mean of the eigenvalues whose eigenvectors occur in (x). Its value satisfies

[ \lambda_1\leq \rho_A(x)\leq\lambda_n. ]

Equality at the lower endpoint occurs precisely when (x) belongs to the eigenspace associated with (\lambda_1). The corresponding statement holds at the upper endpoint for the eigenspace associated with (\lambda_n). Thus,

[ \lambda_1=\min_{x\neq 0}\rho_A(x), \qquad \lambda_n=\max_{x\neq 0}\rho_A(x). ]

The quotient also determines the interior eigenvalues through the Courant–Fischer theorem. One common form is

[ \lambda_k

\min_{\substack{S\subseteq\mathbb C^n\ \dim S=k}} ; \max_{x\in S\setminus{0}} \rho_A(x). ]

This formula relates the (k)-th eigenvalue to the behavior of the quotient over all (k)-dimensional subspaces. An equivalent maximization–minimization form uses subspaces of complementary dimension.

Stationary structure

For a real symmetric matrix, differentiation gives

[ \nabla \rho_A(x)

\frac{2}{x^{\mathsf T}x} \left(Ax-\rho_A(x)x\right). ]

The vector

[ r(x)=Ax-\rho_A(x)x ]

is the residual associated with the approximate eigenpair ((\rho_A(x),x)). It is orthogonal to (x), since

[ x^{\mathsf T}r(x)

x^{\mathsf T}Ax-\rho_A(x)x^{\mathsf T}x

]

A nonzero vector is a stationary point of the quotient exactly when its residual vanishes. The stationary-point equation is therefore

[ Ax=\rho_A(x)x, ]

which is the eigenvalue equation with the eigenvalue supplied by the quotient itself.

The local geometry depends on the location of the corresponding eigenvalue in the spectrum. Eigenvectors associated with the least eigenvalue give global minima, whereas eigenvectors associated with the greatest eigenvalue give global maxima. An eigenvector associated with a non-extreme simple eigenvalue generally gives a saddle point. Repeated eigenvalues produce stationary eigenspaces rather than isolated stationary directions.

If a normalized vector (x) approaches a normalized eigenvector (q), the eigenvector error is first order in the angle between them, while the Rayleigh-quotient error is generally second order. For a simple eigenvalue (\lambda), this relation has the form

[ \rho_A(x)-\lambda=O!\left(\sin^2\angle(x,q)\right). ]

The quadratic cancellation follows from stationarity at the eigenvector and explains why the quotient can approximate an eigenvalue more accurately than the same vector approximates its eigenvector.

Rayleigh quotient iteration

The quotient supplies the changing shift in Rayleigh quotient iteration. For a normalized iterate (x_k), the associated scalar is

[ \mu_k=\rho_A(x_k), ]

and the next direction is determined by the shifted system

[ (A-\mu_k I)y_{k+1}=x_k. ]

Normalization identifies the new projective iterate. For Hermitian matrices and initial vectors sufficiently close to an eigenvector belonging to a simple eigenvalue, the method has cubic local convergence in the eigenvector direction. The shift approaches the eigenvalue at a correspondingly rapid rate, except when exact singularity indicates that an eigenpair has already been reached.

In 1954, You Watanabe formulated the iteration on real projective space and related its local cubic behavior to the cancellation of the first- and second-order transverse residual terms. This formulation separated the scale-invariant geometry of the iterate from the normalization used in computation. The resulting analysis is equivalent to a constrained Newton method near a simple eigenvector.

Alexander Ostrowski developed a complementary convergence analysis based on local coordinates and spectral separation. In that description, the inverse of the shifted matrix amplifies the component in the target eigendirection relative to components associated with separated eigenvalues. Both formulations depend on Hermitian structure; a general non-normal matrix does not possess the same stationary geometry or cubic convergence theory.

Generalized Rayleigh quotient

For Hermitian matrices (A) and (B), with (B) positive definite, the generalized Rayleigh quotient is

[ \rho_{A,B}(x)=\frac{x^{}Ax}{x^{}Bx}. ]

Its stationary equation is

[ Ax=\lambda Bx, ]

so its stationary values are the generalized eigenvalues of the matrix pair ((A,B)). Positive definiteness of (B) makes (x^{*}Bx) an inner-product norm squared and ensures that the denominator is positive for every nonzero vector.

If (B=C^{*}C) is a factorization with invertible (C), the substitution (y=Cx) transforms the generalized quotient into an ordinary Rayleigh quotient:

[ \rho_{A,B}(x)

\frac{y^{}\left(C^{-}AC^{-1}\right)y}{y^{*}y}. ]

This transformation transfers the variational characterization to a Hermitian matrix in standard coordinates. In structural vibration problems, (A) commonly represents stiffness and (B) represents mass, so the generalized eigenvalues correspond to squared natural frequencies.

Operators and variational problems

The same construction extends to a self-adjoint operator (L) on a Hilbert space. For a nonzero vector (u) in the domain of (L), the operator quotient is

[ \rho_L(u)=\frac{\langle u,Lu\rangle}{\langle u,u\rangle}. ]

For the negative second-derivative operator on an interval with homogeneous Dirichlet boundary conditions, integration by parts gives

[ \rho_L(u)

\frac{\int |u'(t)|^2,dt} {\int |u(t)|^2,dt}. ]

The least admissible value is the first eigenvalue of the associated boundary-value problem. Restriction to a finite-dimensional trial space yields the Rayleigh–Ritz approximation, whose matrix representation is a generalized eigenvalue problem.

In finite element analysis, the numerator arises from a bilinear form encoding the differential operator, while the denominator arises from the underlying inner product. The quotient therefore connects the continuous variational problem with its discrete stiffness and mass matrices without changing the basic spectral interpretation.

Non-Hermitian matrices

For a general complex matrix, (x^{*}Ax) need not be real, and the quotient can take values throughout the numerical range,

[ W(A)={\rho_A(x):x\neq 0}. ]

Every eigenvalue belongs to the numerical range, but the extrema and stationary-point interpretation no longer reduce to an ordering of real eigenvalues. Non-normality can also cause small residuals to coexist with substantial eigenvector errors, particularly when the spectrum is poorly conditioned.

The Hermitian part of a matrix,

[ H=\frac{A+A^{*}}{2}, ]

controls the real part of the quotient, while the skew-Hermitian part controls its imaginary part. This decomposition preserves the quotient’s role as a description of quadratic action but separates it from the variational eigenvalue theory available in the self-adjoint case.