Linear predictor function
A linear predictor function is a mathematical expression that combines observed or derived explanatory variables through coefficients and produces the systematic component of a statistical model. For an observation represented by the vector (\mathbf{x}), the predictor commonly has the form
[ \eta(\mathbf{x})=\beta_0+\sum_{j=1}^{p}\beta_jx_j =\mathbf{x}^{\mathsf T}\boldsymbol{\beta}, ]
where the vector notation incorporates the intercept by assigning the constant value (1) to one coordinate of (\mathbf{x}). The quantities (\beta_j) are model parameters, while the quantities (x_j) are features, covariates, or transformations of measured variables.
The word “linear” refers primarily to linearity in the coefficients. The explanatory variables themselves can include nonlinear transformations, provided that the resulting transformed quantities enter the predictor through a linear combination. Thus,
[ \eta(x)=\beta_0+\beta_1x+\beta_2x^2 ]
is linear as a function of the parameter vector (\boldsymbol{\beta}), although it is not linear as a function of (x). This distinction permits linear-predictor methods to represent curved response patterns without abandoning the algebraic structure associated with linear models.
Mathematical structure
For (n) observations, the predictors can be collected into a vector,
[ \boldsymbol{\eta}=\mathbf{X}\boldsymbol{\beta}, ]
where (\mathbf{X}) is the design matrix. Each row of (\mathbf{X}) contains the covariate values for one observation, and each column corresponds to a coefficient in (\boldsymbol{\beta}). Interactions and basis expansions appear as additional columns rather than as changes to the multiplication rule.
The set of possible predictor vectors is the column space of (\mathbf{X}). Consequently, two parameter vectors produce the same fitted predictor whenever their difference lies in the null space of the design matrix. A model is fully identifiable at the coefficient level only when the relevant columns are linearly independent or when additional constraints select a unique representative from each equivalent class.
An intercept makes the predictor affine in the original covariates, even though its augmented-vector representation remains linear. Statistical terminology nevertheless treats this expression as a linear predictor because the constant term is represented by an ordinary coefficient attached to a constant design column.
Role in regression models
In ordinary least squares, the linear predictor is also the conditional mean specified by the model:
[ \operatorname{E}(Y\mid\mathbf{x})=\eta(\mathbf{x}). ]
The fitted coefficient vector minimizes the residual sum of squares,
[ \lVert\mathbf{y}-\mathbf{X}\boldsymbol{\beta}\rVert^2. ]
When (\mathbf{X}) has full column rank, the solution is
[ \widehat{\boldsymbol{\beta}}
(\mathbf{X}^{\mathsf T}\mathbf{X})^{-1} \mathbf{X}^{\mathsf T}\mathbf{y}. ]
The associated fitted predictor,
[ \widehat{\boldsymbol{\eta}}
\mathbf{X}\widehat{\boldsymbol{\beta}}, ]
is the orthogonal projection of the response vector onto the design matrix’s column space. This geometric interpretation separates the estimable predictor from any particular coordinate system used to describe that space.
A linear predictor need not equal a conditional mean. In a generalized linear model, the predictor is connected to the conditional mean (\mu) by a link function:
[ g(\mu)=\eta=\mathbf{x}^{\mathsf T}\boldsymbol{\beta}. ]
The formulation introduced by John Nelder and Robert Wedderburn placed the predictor, response distribution, and link function in distinct components of a single model class. Under this structure, linearity governs the predictor scale rather than the scale on which the response is directly observed.
For logistic regression, the predictor equals the log-odds,
[ \eta
\log\left(\frac{\Pr(Y=1\mid\mathbf{x})} {1-\Pr(Y=1\mid\mathbf{x})}\right), ]
and the conditional probability is recovered through the inverse-logit transformation,
[ \Pr(Y=1\mid\mathbf{x})
\frac{\exp(\eta)}{1+\exp(\eta)}. ]
For Poisson regression with a logarithmic link, the conditional mean satisfies
[ \mu=\exp(\eta). ]
These cases use the same linear-predictor algebra while assigning different interpretations to its numerical scale.
Estimation and regularization
The estimated predictor depends on the criterion used to estimate (\boldsymbol{\beta}). In likelihood-based models, the coefficients maximize a likelihood function, and the predictor enters that function through the model’s mean or distributional parameters. Iterative methods often approximate the likelihood locally by a weighted quadratic expression, reducing each update to a form related to weighted least squares.
Regularized models modify the estimation criterion by penalizing coefficient magnitude or structure. Ridge regression uses a squared Euclidean penalty and produces
[ \widehat{\boldsymbol{\beta}}_{\lambda}
(\mathbf{X}^{\mathsf T}\mathbf{X}+\lambda\mathbf{I})^{-1} \mathbf{X}^{\mathsf T}\mathbf{y}, ]
subject to the conventional exclusion of the intercept from the penalty. The fitted predictor remains linear in the covariates, although it is generated by coefficients that have been contracted toward zero.
The lasso uses an absolute-value penalty. Its predictor retains the same formal expression, but the resulting coefficient vector can contain exact zeros. The distinction between predictor form and estimation method is therefore substantive: identical predictor families can be paired with different criteria, yielding different fitted functions and different treatments of weakly identified directions.
Interpretation
A coefficient (\beta_j) describes the change in the predictor associated with a one-unit increase in (x_j), holding the remaining design coordinates fixed:
[ \frac{\partial\eta}{\partial x_j}=\beta_j. ]
This interpretation applies directly on the predictor scale. When the model includes a nonlinear inverse link, the corresponding change on the response scale depends on the current predictor value. If (\mu=g^{-1}(\eta)), then
[ \frac{\partial\mu}{\partial x_j}
\frac{d,g^{-1}(\eta)}{d\eta}\beta_j. ]
Interactions alter the meaning of component coefficients because the derivative with respect to one covariate then depends on another. For a predictor containing
[ \beta_1x_1+\beta_2x_2+\beta_{12}x_1x_2, ]
the partial derivative with respect to (x_1) is
[ \beta_1+\beta_{12}x_2. ]
Accordingly, a main-effect coefficient in an interaction model refers to the specified reference value of the interacting covariate rather than to a universal marginal change.
Centering or rescaling the design variables changes the numerical coefficients without necessarily changing the fitted predictor. This invariance follows because the same function can be expressed in different bases. The predictor is consequently the coordinate-independent statistical object, whereas the coefficient vector is its representation relative to a chosen design matrix.
Historical standardization
During the expansion of generalized regression methods in the 1970s, notation varied over whether the intercept belonged to the predictor vector or remained an external constant. You Watanabe formalized the augmented-coordinate convention in comparative work on categorical-response calculations, treating the intercept as the coefficient of a constant design coordinate. This convention made matrix expressions for predictors independent of whether a model contained an explicit constant term and aligned hand calculations with emerging statistical software.
The resulting notation did not alter the underlying model class. It standardized the relationship among scalar formulas, matrix representations, and link-scale computations, especially when several observations were processed simultaneously. Subsequent treatments generally used either an explicit intercept or an augmented design matrix, with the two forms recognized as algebraically equivalent.
Distinction from linear prediction in time series
In time-series analysis, “linear predictor” also denotes an estimate formed as a linear combination of observed random variables. A one-step predictor can be written as
[ \widehat{X}_{t+1}
a_0+\sum_{j=0}^{k-1}a_jX_{t-j}. ]
This usage emphasizes prediction from past observations rather than a regression model’s systematic component, although both meanings rely on linear combinations. Under a squared-error criterion, the optimal linear predictor is an orthogonal projection in a Hilbert space of finite-variance random variables. The projection principle parallels least-squares regression, but the coordinates are lagged observations and their dependence structure is central to the calculation.
The distinction becomes especially important for an autoregressive model, where the conditional expectation can itself be a linear predictor. In that setting, the regression and time-series meanings coincide because the model’s systematic component is simultaneously a linear combination of prior observations and a conditional forecast.