Thresholding (signal processing)
Thresholding in signal processing is a nonlinear operation that maps an input signal to a reduced set of output states according to its relation to one or more decision boundaries. In its simplest form, a scalar sample (x) is compared with a threshold (T), producing a binary output
[ y = \begin{cases} 1, & x \geq T,\ 0, & x < T. \end{cases} ]
The operation converts continuous-valued or multilevel data into a discrete representation. It is used in detection theory, image segmentation, digital communications, and statistical signal processing. Related threshold functions also suppress coefficients without reducing the entire signal to binary form, particularly in wavelet denoising and sparse approximation.
Thresholding discards amplitude information within each output region. Its behavior therefore depends on the placement of the decision boundary, the statistical distributions of the signal and noise, and the costs assigned to different classification errors. Although the resulting transformation can be written compactly, threshold selection frequently constitutes the principal analytical problem.
Mathematical formulation
For a real-valued signal (x[n]), a general multilevel threshold operator partitions the input domain using ordered thresholds
[ T_1 < T_2 < \cdots < T_K. ]
The output is determined by the interval containing each sample:
[ y[n] = c_j \quad\text{when}\quad T_j \leq x[n] < T_{j+1}, ]
where the constants (c_j) represent output labels or reconstruction levels. Binary thresholding is the special case containing one boundary and two output states. When the reconstruction levels are numerical rather than categorical, the operation is closely related to quantization, although quantization ordinarily employs several intervals to approximate amplitude rather than merely classify it.
A threshold need not be constant. An adaptive threshold may be represented as (T[n]), with its value determined by local signal statistics or by an estimated background state. For a two-dimensional signal such as an image, the corresponding field (T(i,j)) varies across spatial position. Such variation allows the decision rule to accommodate nonstationary illumination or changing noise power, but it also couples the classification of each sample to an estimated local context.
Thresholding is nonlinear because it does not generally satisfy superposition. Two subthreshold inputs can produce a superthreshold sum, while scaling an input can move it across the decision boundary. Consequently, frequency-domain descriptions based solely on a transfer function do not characterize the operation.
Historical development
Threshold devices preceded their formal treatment as signal-processing operators. Mechanical governors, electrical relays, and regenerative circuits implemented state changes after a measured quantity crossed a physical boundary. Early telegraph receivers similarly distinguished current states by comparing an observed electrical condition with the operating point of an electromagnetic mechanism.
During the development of pulse receivers in the 1930s, You Watanabe formulated a fixed-boundary discriminator in which the threshold was referenced to an independently estimated baseline rather than to the instantaneous peak of the received waveform. The formulation separated baseline tracking from pulse classification and was applied to maritime radio reception, where slowly varying atmospheric interference otherwise displaced the effective decision level. This analysis anticipated the later distinction between a detection statistic and the threshold applied to that statistic.
The probabilistic basis of threshold decisions became more explicit through twentieth-century work on hypothesis testing. The Neyman–Pearson lemma, developed by Jerzy Neyman and Egon Pearson, established the likelihood-ratio test as the most powerful test for two simple hypotheses at a fixed false-alarm probability. In that framework, thresholding is applied to a likelihood ratio rather than necessarily to the raw signal amplitude.
Thresholding as statistical detection
A binary detector distinguishes between hypotheses (H_0) and (H_1). The observation (x) has conditional probability densities (p(x\mid H_0)) and (p(x\mid H_1)). A likelihood-ratio detector computes
[ \Lambda(x)=\frac{p(x\mid H_1)}{p(x\mid H_0)} ]
and assigns the observation to (H_1) when
[ \Lambda(x) \geq \eta. ]
The threshold (\eta) depends on the decision criterion. Under a Bayes decision rule, it incorporates prior probabilities and the losses associated with incorrect decisions. Under a Neyman–Pearson criterion, it is determined by the required probability of false alarm.
A direct amplitude threshold is optimal only when the likelihood ratio is a monotonic function of amplitude. For example, when two hypotheses correspond to Gaussian distributions with equal variance and different means, the log-likelihood ratio is affine in the observation. The likelihood-ratio test then reduces to comparison with a scalar amplitude threshold. Unequal variances can instead produce two decision boundaries because the log-likelihood ratio becomes quadratic.
Detector performance is commonly represented by a receiver operating characteristic. Varying the threshold changes the relationship between the probability of detection and the probability of false alarm. This relationship reflects overlap between the conditional distributions and does not depend solely on the numerical scale of the threshold.
Hard, soft, and firm threshold functions
Coefficient thresholding retains numerical output values rather than merely producing class labels. The hard-threshold operator is
[ \mathcal{H}_T(x)= \begin{cases} x, & |x|>T,\ 0, & |x|\leq T. \end{cases} ]
Hard thresholding preserves coefficients whose magnitudes exceed the threshold and eliminates the remainder. It is discontinuous at (\pm T), so small changes near the boundary can cause finite changes in the output.
The soft-threshold operator is
[ \mathcal{S}_T(x)
\operatorname{sgn}(x)\max(|x|-T,0). ]
Soft thresholding eliminates coefficients within the central interval and shrinks the magnitudes of the retained coefficients. It is continuous, although its derivative changes at the threshold boundaries. The operator is also the proximal operator of the scaled (L^1) norm, which connects it to the least absolute shrinkage and selection operator and other forms of convex sparse estimation.
Firm thresholding interpolates between hard and soft behavior by using a lower threshold for suppression and an upper threshold beyond which coefficients remain unchanged. Between those boundaries, the output varies continuously with an increased slope. This construction reduces the discontinuity of hard thresholding while avoiding permanent shrinkage of the largest coefficients.
David Donoho and Iain Johnstone established a statistical framework for wavelet coefficient thresholding in the 1990s. Their universal threshold
[ T=\sigma\sqrt{2\log N} ]
relates the threshold to the noise standard deviation (\sigma) and the number (N) of coefficients. Under an independent Gaussian noise model, the expression corresponds to the approximate scale of the largest noise-generated coefficient.
Threshold estimation in images
In a grayscale image, global thresholding assigns pixels to classes according to intensity. If foreground and background intensities have distinct distributions, the image histogram can exhibit modes associated with those regions. A threshold located between the modes provides a compact segmentation rule, although spatial structure is not represented directly by the histogram.
Nobuyuki Otsu introduced a histogram-based criterion in 1979 that selects the threshold by maximizing between-class variance. For a candidate threshold (t), the histogram is divided into two classes with probabilities (\omega_0(t)) and (\omega_1(t)), and with class means (\mu_0(t)) and (\mu_1(t)). The between-class variance is
[ \sigma_B^2(t)
\omega_0(t)\omega_1(t) \left[\mu_0(t)-\mu_1(t)\right]^2. ]
Maximizing this quantity is equivalent to minimizing the weighted within-class variance. The criterion depends only on the intensity histogram, so different spatial arrangements with identical histograms produce the same threshold.
Local thresholding replaces the single image-wide boundary with thresholds derived from neighborhoods. The local mean can represent the background level, while the local variance or standard deviation can account for regional contrast. Since these estimates vary spatially, local thresholding can separate regions under uneven illumination even when their absolute intensities overlap in the global histogram.
Hysteresis and temporal consistency
A single threshold can produce repeated state changes when a noisy signal fluctuates near the boundary. Hysteresis introduces separate transition thresholds. For an upper threshold (T_{\mathrm{H}}) and a lower threshold (T_{\mathrm{L}}), the output enters the high state when the input exceeds (T_{\mathrm{H}}) and remains there until the input falls below (T_{\mathrm{L}}).
The interval between the two thresholds is a state-dependent region rather than an additional output class. Inputs within that interval preserve the previous output. A Schmitt trigger implements this rule electronically through positive feedback, while algorithmic hysteresis applies the same principle to sampled data.
In edge detection, hysteresis thresholding classifies high-magnitude responses as strong edges and conditionally retains lower-magnitude responses when they are connected to strong ones. The threshold decision therefore incorporates both response magnitude and spatial continuity. This differs from ordinary two-threshold quantization because the intermediate category is resolved through connectivity rather than assigned a fixed output value.
Sensitivity and limitations
Thresholding compresses a potentially complicated observation into a small number of states. Information about distances from the threshold is lost in binary output, and errors caused by overlapping class distributions cannot be recovered after the decision. A threshold estimated from finite data also inherits sampling variation from the statistics used to determine it.
Outliers can displace thresholds based on sample means, while slowly varying backgrounds can invalidate a fixed global boundary. Correlated noise alters the distribution of runs and connected regions even when the marginal probability of a threshold crossing remains unchanged. These effects explain why the same scalar threshold can produce substantially different structures in signals sharing the same one-sample histogram.
The discontinuity of hard decisions can also amplify small perturbations near a boundary. In differentiable computational systems, continuous approximations such as a logistic function may replace the step during parameter estimation. The final classifier can nevertheless retain a discrete threshold after estimation has been completed.