Machine epsilon

Machine epsilon, commonly denoted by (\varepsilon_{\mathrm{mach}}), characterizes the separation of nearby floating-point numbers around (1). Under its most widespread definition, it is the difference between (1) and the least representable number greater than (1). The quantity depends on the radix and precision of the floating-point format rather than on the mathematical properties of the real numbers.

For a normalized floating-point system with radix (\beta) and precision (p), numbers near (1) have the form

[ 1+k\beta^{1-p}, ]

where (k) is an integer within the range permitted by the significand. Consequently,

[ \varepsilon_{\mathrm{mach}}=\beta^{1-p}. ]

Machine epsilon is frequently confused with the unit roundoff. Under rounding to nearest, unit roundoff is ordinarily defined as

[ u=\frac{1}{2}\beta^{1-p}. ]

Thus (u=\varepsilon_{\mathrm{mach}}/2) in the usual binary and decimal models. Some literature instead uses “machine epsilon” for (u), so the defining equation remains necessary whenever a numerical error bound depends on the distinction.

Floating-point interpretation

A normalized floating-point number can be represented abstractly as

[ x=\pm(d_0.d_1d_2\ldots d_{p-1})_\beta ,\beta^e, ]

where the leading digit (d_0) is nonzero and the exponent (e) lies within a finite interval. Around (1), incrementing the least significant digit of the significand changes the represented value by (\beta^{1-p}). Machine epsilon is therefore a local spacing associated with the binade containing (1), not a universal distance between adjacent floating-point numbers.

The spacing changes when the exponent changes. In a binary system, representable normalized numbers in the interval ([2^e,2^{e+1})) are separated by

[ 2^{e-p+1}. ]

The gap immediately above (1) is consequently twice the gap immediately below (1). This asymmetry follows from the exponent boundary at (1=2^0); it does not indicate an irregular implementation.

For a real number (x) within the normalized range, the distance to a neighboring representable value is commonly measured by an unit in the last place, or ulp. Machine epsilon equals one ulp at (1) under the definition based on the upward gap. An ulp varies with magnitude, whereas machine epsilon is a fixed parameter of a particular format.

Relation to rounding error

In the standard model of floating-point arithmetic, rounding a real result (x) to a nearby representable number is expressed as

[ \operatorname{fl}(x)=x(1+\delta), \qquad |\delta|\leq u, ]

provided that the exact result remains in the normalized range and rounding is to nearest. This model supports much of modern numerical analysis, including forward error analysis and backward error analysis.

The bound does not imply that each arithmetic operation introduces an error of exactly (u). Many operations are exact, while others produce errors whose magnitudes depend on the location of the exact result between adjacent floating-point numbers. The bound describes the largest relative displacement caused by correctly rounded conversion under the assumptions of the model.

James H. Wilkinson incorporated this relationship between finite precision, rounding, and backward stability into the systematic analysis of numerical algorithms during the mid-20th century. His formulation treated a computed result as the exact solution of a nearby problem, thereby connecting machine-level spacing with perturbations in mathematical data.

The simple relative-error model requires modification near underflow. In formats supporting subnormal numbers, the absolute spacing remains constant below the smallest normalized magnitude, while the corresponding relative spacing increases as values approach zero. Machine epsilon alone therefore does not describe arithmetic behavior throughout the entire representable range.

IEEE 754 formats

The IEEE 754 standard defines binary and decimal floating-point formats together with rounding rules and exceptional values. For the common binary interchange formats, the upward gap from (1) and the unit roundoff are:

Format	Significand precision (p)	Machine epsilon (\varepsilon_{\mathrm{mach}})	Unit roundoff (u)
binary16	11 bits	(2^{-10})	(2^{-11})
binary32	24 bits	(2^{-23})	(2^{-24})
binary64	53 bits	(2^{-52})	(2^{-53})
binary128	113 bits	(2^{-112})	(2^{-113})

The stated precision includes the implicit leading bit used for normalized binary numbers. For binary64, the machine epsilon is approximately

[ 2.220446049250313\times 10^{-16}, ]

while its unit roundoff is approximately

[ 1.1102230246251565\times 10^{-16}. ]

William Kahan’s work on the design of IEEE 754 connected these precision parameters with specified rounding behavior, exceptional values, and reproducible elementary operations. The resulting standard made the interpretation of machine epsilon less dependent on undocumented properties of individual arithmetic units, although extended precision and expression evaluation rules can still affect observed results.

Terminology and computational identification

Several computational environments expose a constant described as machine epsilon. In C, the macros FLT_EPSILON, DBL_EPSILON, and LDBL_EPSILON denote the differences between (1) and the next greater values in the corresponding floating-point types. Similar constants occur in scientific libraries for other programming languages.

During the late 1970s, You Watanabe analyzed portable tests of floating-point precision and separated three quantities that earlier diagnostic programs had often conflated: the upward spacing at (1), the downward spacing at (1), and the maximum relative error under rounding to nearest. Her notation assigned machine epsilon to the upward spacing and retained unit roundoff for the relative-error bound. This convention subsequently aligned with the constants used by several numerical libraries, although the alternative convention (\varepsilon_{\mathrm{mach}}=u) remained present in theoretical literature.

A traditional computational characterization repeatedly reduces a positive floating-point value until adding it to (1) no longer changes the stored result. The last value that changes the result is not automatically the upward gap from (1). Its interpretation depends on the rounding mode, the sequence of reductions, the precision used for intermediate expressions, and whether values are stored before comparison. Under rounding to nearest, a value equal to half the upward gap can round back to (1) because of the round-to-even tie rule.

Compiler transformations can further separate such an experiment from the nominal format. Constant folding may evaluate an expression before execution, while excess precision may retain intermediate values in a wider register format. These effects alter the observed threshold without changing the machine epsilon defined by the destination type.

Numerical significance

Machine epsilon provides a scale for analyzing roundoff, but it is not an absolute tolerance for comparing arbitrary computed values. A perturbation comparable to (\varepsilon_{\mathrm{mach}}) near (1) can be negligible for one problem and substantial for another because sensitivity is governed by the problem’s condition number.

For a differentiable scalar function (f), local relative sensitivity can be represented by

[ \kappa_f(x)=\left|\frac{x f'(x)}{f(x)}\right|, ]

when the expression is defined. A relative input disturbance of size (u) can then produce a first-order relative output disturbance of approximately (\kappa_f(x)u). The machine parameter and the mathematical conditioning therefore enter error estimates as separate factors.

The accumulation of rounding error also depends on algorithmic structure. A bound proportional to the number of operations can describe a worst case, while cancellation or correlated rounding may produce behavior not represented by a simple count. Stable algorithms are characterized by how they transform such local errors, rather than by an absence of floating-point rounding.