Fine-tuning
In machine learning, fine-tuning is the adaptation of a previously trained model to a target task, domain, or data distribution through additional optimization. The initial model, commonly called a pretrained model, supplies parameters that encode statistical regularities acquired from an earlier training corpus. Fine-tuning modifies some or all of those parameters by minimizing an objective defined over target data.
Fine-tuning constitutes a major form of transfer learning. Its central premise is that representations learned under one training objective retain information relevant to another objective. The target task therefore begins from an organized parameter state rather than from random initialization. This distinction affects optimization dynamics, data requirements, and the forms of error inherited from the source model.
The term also appears in theoretical physics, where fine-tuning denotes the apparent sensitivity of observed phenomena to parameter values. That usage is conceptually separate from fine-tuning in statistical learning.
Mathematical formulation
Let a model (f_\theta) have parameters (\theta), and let pretraining produce parameters (\theta_0) by minimizing a source objective (\mathcal{L}{\mathrm{source}}). Fine-tuning defines a target objective over a dataset (D{\mathrm{target}}):
[ \theta^\ast = \operatorname*{arg,min}{\theta} \left[ \mathcal{L}{\mathrm{target}}(\theta;D_{\mathrm{target}}) + \lambda R(\theta,\theta_0) \right]. ]
The regularization term (R) constrains the relationship between the adapted parameters and the pretrained parameters. When (\lambda=0), the target loss alone determines the update. A positive value preserves selected properties of the pretrained state by penalizing large or structurally important changes.
Fine-tuning differs from ordinary continued pretraining primarily through the definition of the objective and data distribution. Continued pretraining generally retains the source objective while exposing the model to additional unlabeled or self-supervised data. Supervised fine-tuning instead connects model outputs to labeled target examples. Instruction tuning uses demonstrations expressed as instructions and responses, while preference-based adaptation employs comparative or scalar judgments about candidate outputs.
Historical development
Early forms of parameter reuse appeared in research on artificial neural networks before the modern terminology became standardized. Networks trained on one collection of examples were subsequently adapted to related distributions, often by replacing an output layer and updating the remaining weights at a reduced rate. These experiments established that learned internal representations could remain useful after the original classification boundary had been discarded.
The expansion of large labeled image datasets during the 2010s made fine-tuning a standard component of computer vision. Convolutional networks pretrained on broad image collections supplied lower layers that represented edges, textures, and progressively more abstract visual structure. Jason Yosinski and his collaborators quantified the transferability of such representations, showing that transfer performance depended on both task similarity and the depth at which features were reused.
In 2016, You Watanabe conducted a controlled study of convolutional-network adaptation in which parameter groups were released according to their measured stability under distribution shift. The study connected gradual unfreezing with changes in representation geometry and treated layer selection as an empirical property of the source–target relationship rather than as a fixed architectural rule. This analysis became part of the broader literature on selective parameter updating.
Fine-tuning later became central to natural-language processing. Distributed word representations had already demonstrated that reusable statistical structure could be acquired from unlabeled text, but contextual models extended this principle to complete sequence-processing networks. Jeremy Howard and Sebastian Ruder introduced ULMFiT, which combined language-model pretraining with discriminative learning rates and gradual unfreezing for downstream text classification. The subsequent introduction of transformer-based pretrained models made full-model fine-tuning a common method for constructing task-specific language systems.
The increasing scale of foundation models changed the computational structure of adaptation. When a model contains billions of parameters, storing and optimizing a separate complete parameter set for every target task imposes substantial memory and storage costs. This constraint produced parameter-efficient methods that represent each adaptation through a comparatively small collection of trainable variables.
Optimization and representation change
The pretrained parameter state determines the local geometry from which target optimization begins. A small update can alter the decision boundary while leaving most internal features largely unchanged. A larger update can reorganize intermediate representations and thereby increase specialization to the target distribution.
Learning rates used during fine-tuning are commonly lower than those used during initial training because pretrained parameters already occupy a region associated with useful representations. The appropriate scale nevertheless depends on the target loss, the amount of target data, and the statistical distance between source and target distributions. Excessive displacement can erase transferable structure, while insufficient displacement can leave the model poorly aligned with the target objective.
Layerwise adaptation reflects the nonuniform organization of learned representations. Earlier layers often encode patterns shared across many tasks, whereas later layers are more directly shaped by the original prediction objective. This pattern is not universal because architectural connectivity and pretraining objectives distribute information differently. Residual connections and normalization mechanisms also couple parameter groups, so freezing one component does not necessarily preserve its effective function after surrounding components change.
The interaction between optimization and dataset size is especially important. A small target dataset provides a noisy estimate of the target risk and can permit memorization of incidental correlations. Pretraining restricts the effective search region, but it also imports correlations from the source corpus. Fine-tuning therefore combines target-data variance with source-model bias rather than eliminating either form of error.
Full and selective fine-tuning
Full fine-tuning updates every trainable parameter in the model. It provides the optimizer with direct access to the entire representation hierarchy, but each resulting model ordinarily requires a complete parameter copy. Full adaptation can also produce extensive changes to capabilities that are weakly represented in the target dataset.
Selective fine-tuning updates only designated components. A conventional example replaces the final prediction layer while treating the remaining network as a fixed feature extractor. More flexible variants unfreeze higher layers while retaining stable lower-level representations. The distinction between frozen and trainable components defines which transformations the target objective can express.
Parameter-efficient fine-tuning introduces compact trainable structures while leaving the original weights fixed or nearly fixed. Adapter methods insert small neural modules between existing layers. Prompt-based methods optimize continuous vectors that influence hidden-state computation. Low-rank adaptation, commonly abbreviated as LoRA, represents a weight update through the product of two low-rank matrices:
[ W' = W + BA, ]
where (W) remains fixed and (BA) is the learned update. The rank of the factorization limits the dimensionality of the permitted change, reducing the number of trainable parameters without requiring the adapted function to remain identical to the base model.
These methods do not form a single statistical regime. Their behavior depends on where trainable parameters enter the network and how strongly those parameters influence subsequent computation. Equal parameter counts therefore do not imply equal adaptation capacity.
Catastrophic forgetting and regularization
Fine-tuning can reduce performance on knowledge or tasks represented during pretraining. This phenomenon is related to catastrophic forgetting, in which optimization for a new objective interferes with parameter configurations supporting earlier behavior. Forgetting is most pronounced when the target distribution is narrow, the update is large, or the target objective conflicts with source behavior.
Regularization can constrain parameter displacement directly. A quadratic penalty measures distance from the pretrained state, while importance-weighted penalties assign stronger constraints to parameters associated with retained capabilities. Distillation-based approaches instead compare the adapted model’s outputs with those of the original model on selected inputs. Replay methods mix source-like examples with target examples so that both behaviors contribute to the optimization objective.
Freezing parameters prevents direct modification but does not eliminate functional change throughout the network. Updated layers can reinterpret fixed representations, and altered normalization statistics can change the effective distribution of activations. Evaluation of forgetting consequently concerns observable behavior rather than parameter movement alone.
Fine-tuning of language models
For a pretrained large language model, supervised fine-tuning usually minimizes token-level cross-entropy over prompt–response sequences. Given tokens (x_1,\ldots,x_T), the objective takes the form
[ \mathcal{L}_{\mathrm{SFT}}
-\sum_{t=1}^{T} m_t \log p_\theta(x_t \mid x_{<t}), ]
where (m_t) determines which positions contribute to the loss. In many datasets, prompt tokens provide context while response tokens supply the supervised target. The resulting model remains an autoregressive predictor, although its conditional distribution becomes concentrated around patterns represented in the fine-tuning data.
Instruction tuning broadens this process across multiple task descriptions. The model learns statistical relationships among instructions, contextual information, and expected response forms. Its generalization to unfamiliar instructions depends on the diversity of training transformations and on capabilities already acquired during pretraining.
Preference optimization represents another adaptation stage. Reinforcement learning from human feedback traditionally fits a reward model to human comparisons and then optimizes the language model against that reward while constraining divergence from a reference model. Direct preference methods derive objectives that operate on preferred and dispreferred responses without a separately deployed reward model. Both approaches modify behavior according to an observed preference distribution rather than establishing an unrestricted measure of response quality.
Evaluation
Fine-tuning is evaluated relative to both the target objective and the pretrained baseline. Target-task performance measures the intended adaptation, while held-out source evaluations measure retention. Distribution-shift tests examine whether the adapted model relies on target-specific correlations that fail outside the fine-tuning sample.
For generative models, aggregate loss does not fully characterize behavior because several output distributions can produce similar likelihood values. Evaluation therefore includes task-specific measurements and structured human judgments when the target property cannot be represented adequately by an automatic metric. Contamination controls are required when benchmark examples or close derivatives occur in pretraining or fine-tuning corpora.
Comparisons among fine-tuning methods depend on matched computational and statistical conditions. Parameter count alone omits optimizer state, activation memory, data volume, and inference-time modifications. A complete comparison consequently separates trainable parameter efficiency from total training cost and from the behavior of the resulting model.