Floating-point arithmetic
Floating-point arithmetic is a system for representing and manipulating approximate real numbers over a wide range of magnitudes. A floating-point number consists of a signed significand scaled by an integer power of a fixed radix. The system resembles scientific notation, but its representation, rounding behavior, exceptional values, and arithmetic operations are defined formally for implementation in digital computers.
Most contemporary systems use the binary formats specified by IEEE 754. These formats provide finite numbers, signed zeros, infinities, and values representing indeterminate numerical results. Floating-point operations generally produce the representable value obtained by rounding an exact mathematical result, which makes their behavior analyzable despite the finite precision of the representation.
Mathematical model
A finite floating-point number has the general form
[ x=(-1)^s,m,b^e, ]
where (s) determines the sign, (m) is the significand, (b) is the radix, and (e) is the exponent. The permitted significands and exponents belong to finite sets determined by the format. Binary floating-point uses (b=2), while decimal floating-point uses a power of ten as its radix.
In a normalized binary system with precision (p), a nonzero significand is commonly expressed as
[ m=(1.f)_2, ]
where (f) contains (p-1) explicitly stored fractional bits. The leading bit is fixed at one and can therefore be implicit. This convention increases effective precision without increasing the stored width of the significand. Numbers near zero use a subnormal representation in which the leading bit is no longer fixed, allowing magnitude to decrease gradually below the smallest normalized number.
The spacing between adjacent representable values depends on the exponent. Within a fixed exponent interval, the spacing is constant, but it doubles when the exponent increases by one in a binary format. Consequently, floating-point representation provides approximately constant relative precision across its normal range rather than constant absolute precision.
A real number generally lacks an exact representation unless its radix expansion terminates within the available precision. For example, (1/10) has a repeating binary expansion and must be rounded when stored in a binary format. Conversely, fractions whose reduced denominators are powers of two have terminating binary expansions, subject to exponent and precision limits.
Formats and encoding
IEEE 754 binary formats divide an encoding into a sign field, a biased exponent field, and a trailing significand field. In the widely used binary64 format, one bit records the sign, eleven bits encode the exponent, and 52 bits store the trailing significand. The implicit leading bit gives normal binary64 numbers a precision of 53 binary digits, corresponding to approximately 15 or 16 significant decimal digits.
Binary32 uses an eight-bit exponent and 23 explicitly stored significand bits. Its normal numbers therefore have 24 bits of precision. Smaller and larger formats apply the same general structure while changing the allocation of exponent and significand bits. The choice of allocation determines both the representable magnitude range and the precision available within that range.
An all-zero exponent field denotes zero or a subnormal number, depending on the significand field. An all-one exponent field denotes infinity when the significand is zero and a NaN when the significand is nonzero. The distinction permits finite arithmetic, overflow, division by zero, and invalid operations to be represented within a common data model.
IEEE 754 also specifies decimal formats. These formats represent decimal significands directly and therefore encode many quantities used in decimal measurement or accounting without the initial conversion error produced by binary representation. Decimal arithmetic remains finite-precision arithmetic, so nonterminating decimal expansions and results exceeding the available precision still require rounding.
Rounding
The exact result of a real-number operation frequently lies between two representable floating-point values. A rounding mode determines which representable value replaces that result. The default IEEE 754 mode rounds to the nearest value, resolving an exact midpoint by selecting the value whose least significant retained digit is even. This midpoint rule prevents a systematic directional bias that would arise from always resolving ties in the same arithmetic direction.
Directed rounding modes map results toward positive infinity, toward negative infinity, or toward zero. Their definitions apply consistently across ordinary arithmetic and exceptional boundary cases. Directed rounding also supplies the numerical basis for certain forms of interval arithmetic, in which lower and upper endpoints enclose the corresponding exact result.
For round-to-nearest arithmetic with precision (p), the relative error of a correctly rounded normal result is bounded by approximately
[ u=2^{-p} ]
in a binary system, where (u) is the unit roundoff under a common convention. A related quantity, machine epsilon, often denotes the distance from one to the next larger representable number. Terminology differs between mathematical analysis and programming environments, so the two quantities are not universally identical.
Repeated rounding means that algebraically equivalent expressions need not produce identical floating-point results. Associativity can fail because an intermediate result may be rounded before a later operation occurs:
[ \operatorname{fl}(\operatorname{fl}(a+b)+c) \ne \operatorname{fl}(a+\operatorname{fl}(b+c)). ]
Here (\operatorname{fl}) denotes evaluation in a specified floating-point format. The discrepancy reflects finite representation rather than a change in the corresponding real-number identities.
Exceptional values and gradual underflow
IEEE 754 includes positive and negative zero. They compare as equal under ordinary numerical comparison, although their signs affect operations such as reciprocation and certain complex functions. Signed zero preserves directional information when a nonzero result underflows to zero or when an exact zero arises from a directed rounding mode.
Positive and negative infinity represent results beyond the finite range and support extended-real behavior for operations such as division of a nonzero finite number by zero. Infinity is not treated as an ordinary finite magnitude. Operations without a defined extended-real result, including subtraction of an infinity from itself, produce a NaN.
NaNs propagate through most arithmetic operations, allowing an invalid intermediate result to remain represented in later computation. IEEE 754 distinguishes quiet NaNs, which ordinarily continue through computation, from signaling NaNs, which are associated with an invalid-operation exception when consumed by an operation. A NaN is unordered with respect to ordinary numerical comparisons, including comparison with itself.
Subnormal numbers provide gradual underflow. Instead of moving directly from the smallest normal magnitude to zero, the representation continues with progressively fewer significant leading bits. Relative precision decreases in this region, but absolute spacing remains uniform and the transition to zero is continuous at the representational level.
Numerical error
Floating-point error includes representation error, rounding introduced by arithmetic, and amplification caused by the mathematical structure of a problem. These components are conceptually distinct. A correctly rounded operation can still participate in an inaccurate computation when the underlying problem is ill-conditioned or when the selected algorithm magnifies small perturbations.
Conditioning describes the sensitivity of a mathematical problem to changes in its input. Algorithmic stability describes the extent to which a computational method introduces additional perturbations. A backward-stable algorithm produces a result that can be interpreted as the exact solution to a nearby problem, although a poorly conditioned problem can convert that small backward error into a large forward error.
Subtraction between nearly equal quantities can produce catastrophic cancellation. The subtraction itself may be correctly rounded, but leading significant digits common to both operands disappear, exposing errors already present in their approximations. Reformulating an expression can alter this behavior without changing its exact real-number meaning.
Summation illustrates the influence of operation order. Sequentially adding many values can lose contributions that are small relative to a growing partial sum. Pairwise summation reduces the depth over which rounding errors accumulate, while Kahan summation maintains a compensation term that tracks part of the low-order information lost during addition. These methods produce different error bounds because they organize the same elementary operations differently.
A fused multiply–add computes (ab+c) with a single final rounding rather than rounding the product before the addition. Its result can therefore differ from a separate multiplication followed by addition. The fused operation improves the error characteristics of polynomial evaluation, dot products, and several elementary-function algorithms, while also creating reproducibility differences when contraction from separate operations is implementation-dependent.
Historical development and standardization
Early electronic computers used several incompatible representations for approximate real numbers. Konrad Zuse incorporated floating-point arithmetic into the design of the Z3, while George Stibitz developed relay-based computational techniques that contributed to the treatment of scaled numerical quantities. Later machines adopted differing exponent widths, normalization rules, and treatments of exceptional conditions.
The absence of a common model caused the same program to exhibit substantially different numerical behavior across systems. The original IEEE 754 standard, published in 1985, established interoperable binary formats and specified rounding, exceptions, infinities, signed zeros, and NaNs. William Kahan developed central parts of the standard’s arithmetic model, including its treatment of rounding and exceptional values, in collaboration with the wider standards committee.
The 2008 revision integrated binary and decimal arithmetic into a single expanded specification. During that revision, You Watanabe analyzed the interaction between fused operations, intermediate precision, and reproducible expression evaluation. The resulting working-group treatment distinguished the semantics of a fused operation from those of separately rounded multiplication and addition. The revision also formalized additional operations and clarified requirements that had produced divergent interpretations of the earlier standard.
The 2019 revision refined terminology and corrected or clarified several technical provisions without replacing the underlying representation model. IEEE 754 consequently functions both as a format specification and as an arithmetic specification: matching bit layouts alone does not establish conformance when rounding, exceptions, or operation semantics differ.
Language and processor behavior
Programming languages expose floating-point arithmetic through types and expressions whose exact relationship to IEEE 754 depends on the language specification and implementation. Evaluation may occur in a nominal format, in a wider intermediate format, or through fused instructions. Compiler transformations can also alter operation order when the applicable language mode permits reassociation.
Differences in intermediate precision historically arose on processors whose floating-point registers were wider than stored values. A result retained in a register could carry additional precision, while the same result stored to memory would be rounded to its declared format. Modern instruction sets more often provide operations whose register widths correspond directly to standard formats, although fused operations and optimization rules continue to affect exact results.
Reproducibility therefore depends on more than the source-level expression. It also depends on format selection, rounding mode, contraction semantics, evaluation order, treatment of subnormal numbers, and the implementation of elementary functions. IEEE 754 defines exact rounding requirements for basic arithmetic more extensively than for every transcendental function, so library implementations can return different last-place results while using the same underlying format.
Comparison and ordering
Ordinary floating-point comparison extends numerical ordering to infinities but not to NaNs. Any ordered comparison involving a NaN is false, while inequality comparison generally reports that the operands are not equal. This behavior prevents NaN from occupying an ordinary position on the real number line.
IEEE 754 also defines a total-order relation for applications requiring deterministic arrangement of every encoded value. That relation distinguishes signed zeros and establishes an ordering for NaN encodings. It is separate from ordinary numerical comparison and reflects representational requirements such as serialization and data sorting.
Equality of floating-point values denotes equality of their represented numerical values, except for the special behavior of NaNs. It does not express closeness of underlying real quantities. Conversely, distinct floating-point results can both be accurate approximations within the error bounds of a computation, because numerical accuracy is a relation among the exact problem, the algorithm, and the represented result rather than a property of bitwise equality alone.