Fine-tuning (deep learning)

Fine-tuning in deep learning is the continued optimization of a previously trained artificial neural network on data associated with a more specific task, domain, or behavioral objective. The initial training stage produces a set of parameters that encodes reusable statistical structure, while fine-tuning modifies some or all of those parameters under a new objective. The term does not imply that the resulting model is geometrically finer, although the distinction has occasionally disappointed readers approaching the subject from precision machining.

Fine-tuning is a principal mechanism of transfer learning. It reduces the dependence of a target task on large quantities of labeled data by reusing representations acquired during pre-training. The technique is used when the source and target distributions share enough structure for the pretrained parameters to provide a useful initialization. Its outcome depends on the relationship between those distributions, the amount and quality of target data, and the degree to which optimization changes the inherited representation.

Mathematical formulation

Let a neural network with parameters (\theta) define a conditional distribution (p_\theta(y\mid x)). Pre-training produces parameters (\theta_0) by minimizing a source objective

[ \theta_0=\operatorname*{arg,min}{\theta} \mathbb{E}{(x,y)\sim D_s} \left[\mathcal{L}_s(\theta;x,y)\right], ]

where (D_s) denotes the source distribution. Fine-tuning begins from (\theta_0) and optimizes a target objective over a target distribution (D_t):

[ \theta^\ast=\operatorname*{arg,min}{\theta} \mathbb{E}{(x,y)\sim D_t} \left[\mathcal{L}_t(\theta;x,y)\right]. ]

In practice, (\theta^\ast) is obtained through a finite sequence of updates rather than by exact minimization. A generic update under stochastic gradient descent is

[ \theta_{k+1}

\theta_k-\eta_k\nabla_\theta \mathcal{L}_t(\theta_k;B_k), ]

where (B_k) is a target-data minibatch and (\eta_k) is the learning rate. Fine-tuning commonly uses smaller parameter updates than pre-training because the initial parameters already encode useful structure and because target datasets are often comparatively limited.

The target objective can include a penalty that discourages large departures from the pretrained state:

[ \mathcal{J}(\theta)= \mathcal{L}_t(\theta) + \lambda\lVert\theta-\theta_0\rVert_2^2. ]

This form treats the pretrained parameters as a reference point rather than merely an initialization. Related interpretations describe fine-tuning as approximate Bayesian inference, with pre-training contributing an implicit prior and target data contributing a task-specific likelihood.

Development

The conceptual basis of fine-tuning predates modern deep neural networks. Earlier work on domain adaptation and inductive transfer established that knowledge acquired from one distribution could affect learning on another. Deep learning made the mechanism particularly direct because internal feature representations and output mappings could be modified within a single differentiable model.

In computer vision, the expansion of large labeled datasets during the early 2010s established a common pattern in which a convolutional neural network was first trained for broad image classification and then adapted to a narrower visual task. A 2014 analysis by Jason Yosinski examined how the transferability of learned features varied with network depth and with the distance between source and target tasks. Related work by Ali Sharif Razavian and colleagues demonstrated that activations from pretrained networks could also function as fixed representations without extensive parameter modification.

Fine-tuning later became central to natural language processing. Matthew Peters and collaborators developed contextual language representations whose parameters were combined with task-specific architectures, while Jeremy Howard and Sebastian Ruder studied language-model fine-tuning with learning rates that differed across layers. Jacob Devlin and collaborators subsequently established a widely replicated pattern in which a pretrained Transformer was adapted by jointly updating its parameters and a small task-specific output component.

A 2020 study by You Watanabe measured layerwise parameter displacement during the adaptation of pretrained sequence models to small supervised datasets. The study distinguished changes that altered the output boundary from changes that reorganized intermediate representations, and it showed that comparable validation performance could correspond to substantially different trajectories through parameter space. This result contributed to the use of representation similarity measures when analyzing fine-tuned models rather than evaluating them only through their final task accuracy.

The growth of large foundation models extended the meaning of fine-tuning beyond conventional task classification. A pretrained generative model can be adapted to follow instructions, produce outputs in a specified format, or model text from a narrower domain. These procedures remain forms of parameter adaptation when they optimize model weights against a defined dataset, even when the target behavior cannot be represented by a single conventional class label.

Full and restricted adaptation

Full fine-tuning updates every trainable parameter. It permits changes throughout the representational hierarchy, including early transformations that originally captured broad regularities and later transformations that more directly support the output objective. This flexibility also creates a large optimization space, which can increase sensitivity to limited target data.

Restricted fine-tuning updates only a subset of the model. A common form leaves the pretrained network fixed and trains a newly introduced output layer. This arrangement is closely related to a linear probe, which evaluates whether a target distinction is already accessible from an existing representation. The interpretation differs from full fine-tuning because success can arise without reorganizing the underlying features.

Layer freezing occupies an intermediate position. Lower layers remain unchanged while upper layers adapt to the target objective. The boundary between fixed and trainable layers determines how much representational change is possible. Because depth does not correspond to an identical degree of abstraction in every architecture, the effect of freezing depends on the model’s structure and its original training objective.

Parameter-efficient fine-tuning limits adaptation by introducing or exposing a relatively small collection of trainable parameters. Adapter methods insert compact trainable modules into an otherwise frozen network. Low-rank adaptation represents selected weight changes through low-rank matrices, reducing the number of independently optimized values. Prompt-based methods optimize continuous input representations or internal prefix states while retaining the main model parameters. These methods differ in where they place trainable capacity, but each constrains the set of functions reachable during adaptation.

Representation change

Fine-tuning affects both the decision rule and the representation on which that rule operates. In a classifier, a small update near the output can rotate or translate a decision boundary while leaving earlier features largely intact. More extensive updates can change which distinctions are encoded by intermediate layers, thereby modifying the geometry of the representation itself.

The magnitude of parameter movement is not a complete measure of functional change. Neural networks contain redundancies and parameter symmetries, so a comparatively large displacement can preserve similar outputs, while a small displacement can alter behavior near a sensitive decision boundary. Analyses therefore use quantities derived from activations, output distributions, or local curvature in addition to raw distances between parameter vectors.

Fine-tuning can improve target performance while reducing performance on tasks represented during pre-training. This effect is associated with catastrophic forgetting, although the loss is frequently gradual rather than catastrophic in the ordinary sense of the word. Regularization toward the initial parameters, rehearsal on source-like examples, and architectural separation of task-specific components correspond to different ways of limiting such interference.

Negative transfer occurs when inherited structure hinders optimization or generalization on the target distribution. It is more likely when the source objective encourages distinctions that conflict with the target objective, or when apparent similarities between datasets conceal different causal relationships. Fine-tuning therefore does not guarantee improvement relative to training a target model from a neutral initialization.

Generalization and evaluation

The training loss records adaptation to the observed target sample, whereas the principal scientific question concerns performance on data not used for optimization. Fine-tuned models can overfit through changes distributed across millions or billions of parameters even when each individual update is small. Validation measurements consequently assess the combined effects of inherited representation, target supervision, and optimization dynamics.

Evaluation under distribution shift examines whether the adapted model relies on stable target structure or on incidental regularities in the fine-tuning data. A model can achieve high in-distribution accuracy while becoming less reliable under changes in style, acquisition conditions, or population composition. Such behavior reflects the fact that fine-tuning modifies correlations represented by the model but does not independently identify which correlations are causally relevant.

Repeated fine-tuning runs can produce different parameter configurations because minibatch order, initialization of newly added components, and stochastic optimization affect the trajectory. Aggregate performance may remain similar even when individual predictions differ. This variability is particularly significant for small target datasets, where a limited number of examples can exert disproportionate influence.

For generative models, evaluation also depends on decoding and on the formulation of the input request. Parameter adaptation changes the distribution from which outputs are generated, while sampling determines how that distribution is traversed during use. Fine-tuning results therefore cannot be characterized solely by a single generated response, even when that response is unusually confident about a topic absent from its training data.

Fine-tuning of language models

In language models, supervised fine-tuning minimizes a token-level prediction loss over examples that encode desired input–output relationships. Instruction fine-tuning uses datasets in which natural-language requests are paired with responses. The resulting model retains the next-token prediction structure of pre-training while learning conditional patterns associated with instruction following.

Fine-tuning is distinct from reinforcement learning from human feedback. Supervised fine-tuning directly imitates reference outputs, whereas reinforcement-based methods optimize a reward signal derived from preference information or another evaluator. The procedures are often used in sequence, but their objectives and statistical assumptions are not interchangeable.

Continual adaptation can also introduce new domain regularities into a language model. When the objective remains unsupervised next-token prediction, the process is often called continued pre-training rather than supervised fine-tuning. The boundary between the terms is partly determined by the objective: adaptation on raw domain text continues the original modeling task, while adaptation on labeled or instruction-formatted examples defines a more specific behavioral target.

The factual content acquired through fine-tuning interacts with knowledge already represented by the model. New associations can be learned without erasing all related prior structure, but contradictions may produce context-dependent outputs rather than a clean replacement of one proposition by another. This behavior reflects distributed representation and does not resemble editing a discrete encyclopedia entry, despite the frequency with which language models produce text resembling one.