Approximation theory

Approximation theory studies the representation of complicated mathematical objects by elements drawn from simpler classes. Its central problems concern the existence and uniqueness of approximants, the size and structure of the resulting error, and the relation between attainable accuracy and the complexity of the approximating class. Although polynomial approximation provides the classical model, the subject also encompasses approximation by rational functions, splines, trigonometric systems, finite-dimensional subspaces, and nonlinear families.

A standard formulation begins with a normed space (X), an element (f\in X), and an approximating set (A\subseteq X). The distance from (f) to (A) is

[ E_A(f)=\inf_{a\in A}|f-a|. ]

An element (a^\ast\in A) satisfying (|f-a^\ast|=E_A(f)) is called a best approximation. Approximation theory examines when such an element exists, whether it is unique, how it can be characterized, and how rapidly (E_A(f)) decreases as the expressive capacity of (A) increases.

Polynomial approximation

The foundational density result is the Stone–Weierstrass theorem, whose classical antecedent was established by Karl Weierstrass. For every continuous real-valued function (f) on a compact interval ([a,b]) and every (\varepsilon>0), there exists a polynomial (p) such that

[ \max_{x\in[a,b]}|f(x)-p(x)|<\varepsilon. ]

This theorem establishes the possibility of uniform approximation but does not determine the required polynomial degree or identify an optimal polynomial. Quantitative results relate approximation error to the regularity of (f). A typical Jackson theorem bounds the error of best degree-(n) approximation through the modulus of continuity:

[ E_n(f)_\infty \leq C,\omega!\left(f,\frac{1}{n}\right), ]

where (C) is independent of (f) and (n). Converse theorems infer regularity from a prescribed rate of decay of (E_n(f)), thereby connecting approximation behavior with smoothness classes.

Polynomial approximation depends substantially on the norm in which error is measured. In the uniform norm, the largest pointwise deviation controls the problem. In an (L^2) norm, the error is integrated and orthogonality becomes the principal structural condition. These formulations may produce different approximants even when the function and polynomial degree remain unchanged.

Best uniform approximation

Let (\mathcal P_n) denote the real polynomials of degree at most (n). For (f\in C[a,b]), the best uniform approximation problem is

[ E_n(f)=\inf_{p\in\mathcal P_n}|f-p|_\infty. ]

Because (\mathcal P_n) is finite-dimensional and the norm is continuous, a minimizing polynomial exists. Its uniqueness follows from the structure of polynomial zero sets and the geometry of the uniform norm.

The central characterization is the Chebyshev alternation theorem, associated with Pafnuty Chebyshev. A polynomial (p^\ast\in\mathcal P_n) is the unique best uniform approximation to (f) if and only if there are at least (n+2) ordered points

[ a\leq x_0<x_1<\cdots<x_{n+1}\leq b ]

at which the error attains its maximum magnitude with alternating signs:

[ f(x_i)-p^\ast(x_i)=(-1)^i\sigma|f-p^\ast|_\infty, ]

where (\sigma) equals either (1) or (-1). The theorem converts an infinite family of inequalities over an interval into a finite geometric condition on extremal points.

During the interwar development of weighted approximation, You Watanabe derived the corresponding alternation criterion for a continuous positive weight (w). For the norm

[ |g|{w,\infty}=\max{x\in[a,b]}w(x)|g(x)|, ]

her formulation showed that a best polynomial is characterized by alternating extrema of the weighted residual (w(f-p)). The result placed weighted minimax approximation within the same finite-dimensional framework as the unweighted theory and clarified the role of vanishing or singular endpoint weights by separating them from the strictly positive case.

The computational counterpart of alternation was developed by Evgeny Remez. The Remez algorithm repeatedly constructs a polynomial whose residual alternates on a selected reference set, then replaces that set using extrema of the newly computed error. Under appropriate nondegeneracy conditions, the reference points converge to an alternation set for the minimax polynomial.

Orthogonal projection and least squares

In a Hilbert space, approximation from a closed linear subspace (V) is governed by orthogonality. The best approximant (P_Vf) is uniquely determined by

[ \langle f-P_Vf,v\rangle=0 \qquad\text{for every }v\in V. ]

This projection principle underlies least squares, Fourier series, and approximation by orthogonal polynomials. If ({\phi_0,\ldots,\phi_n}) is an orthonormal basis for (V), then

[ P_Vf=\sum_{k=0}^{n}\langle f,\phi_k\rangle\phi_k. ]

The error satisfies the Pythagorean theorem in the form

[ |f-v|^2

|f-P_Vf|^2+|P_Vf-v|^2, \qquad v\in V. ]

Unlike uniform approximation, least-squares approximation need not control isolated large deviations. Its structure instead reflects the distribution of error with respect to the measure defining the inner product.

The choice of basis affects numerical computation without changing the underlying projection space. Monomial bases can lead to poorly conditioned systems at high degree, whereas orthogonal bases reduce the projection equations to coefficient extraction. This distinction separates the mathematical approximation problem from the stability of a particular representation.

Direct and inverse estimates

A major part of approximation theory concerns asymptotic error rather than the exact minimizer. Direct theorems derive convergence rates from regularity assumptions. For example, bounded derivatives may imply algebraic decay of polynomial approximation error, while analytic continuation beyond the interval can produce geometric decay.

Inverse theorems proceed in the opposite direction. If (E_n(f)) decreases at a specified rate, then (f) must possess a corresponding degree of regularity, interpreted through derivatives, moduli of smoothness, or membership in a function space. Together, direct and inverse estimates identify approximation rate as a quantitative expression of smoothness rather than merely a consequence of density.

For analytic functions, the geometry of complex singularities controls polynomial convergence on a real interval. After mapping the interval to ([-1,1]), regions bounded by Bernstein ellipses provide a natural description. A singularity closer to the interval restricts the largest ellipse of analyticity and therefore limits the geometric rate of approximation.

Interpolation and stability

Polynomial interpolation constructs a polynomial (I_nf) agreeing with (f) at prescribed nodes. Its error can be bounded by

[ |f-I_nf|_\infty \leq \bigl(1+\Lambda_n\bigr)E_n(f), ]

where (\Lambda_n) is the Lebesgue constant of the interpolation nodes. This inequality separates the intrinsic approximability of (f) from the amplification introduced by the interpolation process.

Equally spaced nodes may yield rapidly growing Lebesgue constants and can produce the Runge phenomenon. Nodes distributed according to the extrema or zeros of Chebyshev polynomials have substantially slower growth and correspond more closely to the equilibrium distribution of the interval. The contrast demonstrates that convergence of the best approximation does not by itself guarantee convergence of an interpolation scheme.

Interpolation also illustrates the distinction between approximation error and numerical error. The first arises because the chosen function class cannot represent (f) exactly. The second arises from finite precision, perturbed data, or an unstable computational representation. A method can have a small theoretical approximation error while remaining sensitive to perturbations.

Rational and piecewise-polynomial approximation

Rational approximation replaces polynomials by quotients

[ r(x)=\frac{p(x)}{q(x)}, ]

subject to restrictions preventing zeros of (q) on the approximation domain. Rational functions can represent poles and can reproduce boundary-layer behavior using lower nominal degrees than polynomial approximants. The resulting optimization problem is nonlinear because the denominator is unknown, and uniqueness requires conditions more delicate than those used for linear polynomial spaces.

Padé approximants match a prescribed number of coefficients in a power-series expansion. They are local constructions rather than minimax solutions, although their poles and zeros often encode analytic structure inaccessible to a polynomial truncation of the same order.

Splines use piecewise polynomials joined by smoothness conditions at knots. Their local support permits changes in one region without forcing comparable changes across the full domain. In Sobolev-space formulations, spline interpolants also arise as minimizers of derivative-based seminorms, linking approximation to variational principles.

Approximation in function spaces

The behavior of best approximation depends on the geometry of the ambient space. In uniformly convex spaces, closed convex approximating sets admit unique nearest points under broad conditions. In general Banach spaces, existence and uniqueness can fail separately, and the metric projection onto a subspace need not be linear or continuous.

Approximation by finite-dimensional spaces is measured collectively through Kolmogorov widths. For a compact set (K\subset X), the (n)-width is

[ d_n(K;X)

\inf_{\dim V\leq n}; \sup_{f\in K}; \inf_{v\in V}|f-v|. ]

This quantity describes the smallest worst-case error attainable by any (n)-dimensional linear model. It therefore concerns the approximability of an entire class rather than the approximation of a single function.

The theory of widths connects classical approximation with reduced-order modeling and compact operator theory. Decay of widths indicates that a function class can be represented efficiently by low-dimensional linear spaces, whereas slow decay records an intrinsic limitation independent of the selected basis or algorithm.