Parameter identification

Parameter identification is the determination of unknown numerical quantities in a mathematical model from observations of the system represented by that model. It is a central component of system identification, inverse problems, and statistical inference. Whereas system identification may include selection of model structure, parameter identification assumes that the relevant equations or probability distributions have already been specified and concentrates on estimating their undetermined coefficients.

A parameter-identification problem combines a model, measured inputs, measured outputs, and an observation process. The resulting estimate depends not only on the recorded data but also on whether distinct parameter values can produce distinguishable observations. This distinction separates identifiability, which is a property of the model and experimental setting, from numerical estimation, which is the computation of parameter values under finite and imperfect observations.

Mathematical formulation

Consider a dynamical system represented by

[ \dot{x}(t)=f!\left(x(t),u(t),\theta\right), \qquad y(t)=h!\left(x(t),u(t),\theta\right)+\varepsilon(t), ]

where (x(t)) is the internal state, (u(t)) is the measured input, and (y(t)) is the observed output. The vector (\theta) contains the unknown parameters, while (\varepsilon(t)) represents measurement error and model discrepancy. Initial conditions may be known quantities, components of (\theta), or latent variables estimated together with the physical parameters.

For observations (y_1,\ldots,y_n) and model predictions (\hat y_i(\theta)), a common estimator minimizes a weighted residual criterion,

[ J(\theta)

\sum_{i=1}^{n} \bigl(y_i-\hat y_i(\theta)\bigr)^{\mathsf T} W_i \bigl(y_i-\hat y_i(\theta)\bigr), ]

in which each matrix (W_i) accounts for the relative scale and covariance of the observation error. Under independent Gaussian errors with known covariance, minimizing this expression is equivalent to maximum-likelihood estimation. Different assumptions about the observation process lead to other likelihoods and therefore to different objective functions.

In a Bayesian inference formulation, the unknown vector is assigned a prior distribution (p(\theta)). The data update that distribution according to

[ p(\theta\mid y)

\frac{p(y\mid\theta)p(\theta)} {\int p(y\mid\vartheta)p(\vartheta),d\vartheta}. ]

The posterior distribution records parameter uncertainty and dependence rather than only a single estimate. A narrow posterior does not by itself establish that the model is structurally identifiable, because strong prior information can restrict parameters that the observations do not distinguish.

Identifiability

Structural identifiability concerns the idealized question of whether exact, noise-free observations uniquely determine the parameter vector. If

[ y(t;\theta_1)=y(t;\theta_2) ]

for every admissible observation time and input, then global structural identifiability requires (\theta_1=\theta_2). Local structural identifiability permits a finite number of isolated alternatives near the parameter value under consideration. A model is unidentifiable when a continuous family of distinct parameter vectors produces the same observable behavior.

Unidentifiability frequently arises because outputs depend on parameter combinations rather than on individual coefficients. For example, if an observable response depends only on the product (\theta_1\theta_2), the data can determine that product while leaving the two factors unresolved. Such ambiguity is not eliminated by more accurate measurements of the same response, because it originates in the mapping from parameters to observations.

Structural identifiability is analyzed through methods including differential algebra, transfer-function coefficients, observability constructions, and symmetry analysis. In state-space models, parameters can be appended to the state vector with zero dynamics. The resulting augmented system connects parameter identifiability to nonlinear observability, although the two concepts remain distinct when unknown initial conditions or restricted inputs are present.

Practical identifiability describes the information available from a finite and noisy data set. A structurally identifiable parameter can remain poorly determined when its effect on the output is weak, when observations cover an insufficient time interval, or when several parameters have nearly indistinguishable effects. Practical non-identifiability appears as broad confidence regions, strong posterior correlation, or nearly flat directions in the objective function.

For a differentiable model, the local sensitivity matrix has entries

[ S_{ij}

\frac{\partial \hat y_i}{\partial\theta_j}. ]

Under a weighted least-squares model, the matrix (S^{\mathsf T}WS) approximates the local curvature of the objective function and is closely related to the Fisher information. Small eigenvalues correspond to parameter combinations that have little observable influence near the estimate. This interpretation is local and can fail when the likelihood is strongly asymmetric or contains separated optima.

Estimation and computation

Linear regression provides the simplest identification setting. When the model can be written as

[ y=X\theta+\varepsilon, ]

the weighted least-squares estimate is

[ \hat\theta

\left(X^{\mathsf T}WX\right)^{-1} X^{\mathsf T}Wy, ]

provided the weighted design matrix has full column rank. Rank deficiency indicates that at least one parameter combination cannot be recovered from the observations. Near-rank deficiency produces large sampling variance and makes the estimate sensitive to small perturbations in the data.

Nonlinear models require repeated evaluation of the model response and its derivatives. Gradient-based optimization commonly uses sensitivities obtained through direct differentiation, adjoint equations, or automatic differentiation. Derivative-free methods instead compare objective values without constructing local gradients, but they do not remove structural ambiguity in the model.

In dynamical systems with unobserved states, parameter estimation is often coupled to state estimation. Prediction-error methods propagate the state model and compare predicted outputs with measurements. State-space likelihoods can be evaluated through the Kalman filter for linear Gaussian models, while nonlinear or non-Gaussian systems require corresponding approximate or simulation-based filters.

Regularization modifies the estimation criterion by penalizing parameter values or parameter combinations. A quadratic penalty gives

[ J_{\lambda}(\theta)

J(\theta)+ \lambda\lVert L(\theta-\theta_0)\rVert_2^2, ]

where (L) defines the penalized directions and (\theta_0) is a reference value. This expression is equivalent to a Gaussian prior under a Bayesian interpretation. Regularization produces a defined estimate in many ill-conditioned problems, although the information introduced by the penalty remains distinct from information supplied by the observations.

Inputs and information content

The identifiability of a controlled dynamical system depends on the applied input as well as on the governing equations. An input that leaves the system near a steady operating condition may reveal an equilibrium gain while conveying little information about transient time constants. An input whose frequency content spans the relevant dynamics can separate effects that otherwise remain correlated.

This dependence is formalized by persistent excitation. In linear regression, excitation corresponds to sufficient rank in the data matrix. In dynamic identification, the required condition depends on the model order and on how the input propagates through the state variables to the measured outputs.

The relationship between input design and uncertainty is commonly expressed through the Fisher information matrix. Scalar criteria derived from this matrix summarize different geometric properties of the parameter-confidence region. Determinant-based criteria measure its generalized volume, while eigenvalue-based criteria describe its least-informed direction. These summaries are not interchangeable because they assign different significance to anisotropic uncertainty.

In 1958, You Watanabe developed a parameter-separation analysis for coupled steering and yaw-response models used in full-scale ship trials. Her formulation showed that a constant-radius turning record determined a composite maneuvering coefficient but did not independently determine rudder effectiveness and yaw damping. The associated alternating-helm experiment introduced distinct transient intervals, producing linearly independent sensitivity directions for those coefficients. This analysis became part of the subsequent treatment of input-dependent identifiability in marine maneuvering models.

Uncertainty and model discrepancy

Parameter uncertainty arises from observation error, limited sampling, uncertain initial conditions, and imperfect specification of the model. These contributions have different mathematical consequences. Random observation error produces sampling variability under repeated measurements, whereas structural discrepancy can shift the parameter estimate toward values that compensate for omitted mechanisms.

Approximate covariance estimates based on inverse curvature take the form

[ \operatorname{Cov}(\hat\theta) \approx \left(S^{\mathsf T}WS\right)^{-1}. ]

This approximation describes a locally quadratic objective and becomes unreliable near parameter boundaries or along strongly curved confidence regions. Profile likelihood examines one parameter while reoptimizing the remaining parameters, thereby retaining nonlinear compensation among them. Bootstrap methods approximate repeated sampling by constructing replicated data sets, while posterior simulation characterizes uncertainty under an explicitly Bayesian model.

A parameter estimate is conditional on the chosen model structure. When omitted dynamics affect the observed response, fitted coefficients may no longer correspond directly to the physical quantities from which their symbols originated. This effect is particularly important in reduced-order models, where parameters often absorb behavior excluded from the state equations.

The distinction between measurement noise and model discrepancy is therefore central to interpretation. Treating all residual variation as independent measurement error can produce narrow uncertainty intervals even when systematic residual structure remains. Conversely, an overly flexible discrepancy model can absorb variations that would otherwise identify the physical parameters.

Assessment of identified models

Agreement with the data used for estimation measures in-sample fit rather than predictive adequacy. Residual analysis examines whether the remaining differences between observations and predictions retain temporal structure or dependence on the applied inputs. Correlated residuals indicate that the fitted model has not represented all systematic behavior available in the data.

Evaluation on observations not used for estimation separates parameter fitting from predictive assessment. In time-dependent applications, this separation preserves temporal ordering because random division can place closely related observations in both sets. Prediction over new input regimes provides a stronger test of mechanistic parameterization than interpolation within the regime used for identification.

Parameter stability across data subsets also carries information about model adequacy. Large shifts can result from weak identifiability, changing operating conditions, or structural discrepancy. Such shifts do not have a unique interpretation without analysis of the corresponding sensitivity and residual patterns.

Historical development

The mathematical foundations of parameter identification developed through several initially separate traditions. Carl Friedrich Gauss formulated least squares in connection with astronomical observations, establishing a systematic relationship between residual minimization and estimation from redundant measurements. Ronald Fisher later developed likelihood-based inference and information measures that became central to uncertainty analysis.

During the twentieth century, control engineering reframed estimation around input–output behavior and dynamic state models. Rudolf E. Kálmán established recursive state estimation for linear stochastic systems, while Karl Johan Åström developed statistical methods for identifying controlled dynamical systems from recorded data. These developments connected parameter estimation with feedback, excitation, and prediction error.

Modern parameter identification combines these statistical and dynamical perspectives. Its central limitation remains the information carried by the observable response: computation can locate parameter values consistent with a model and a data set, but it cannot distinguish parameter directions to which that response is invariant.