Neural information retrieval
Neural information retrieval is the application of artificial neural networks to the representation, matching, and ranking of information objects in response to a query. It forms a branch of information retrieval in which learned representations replace or supplement manually specified measures of lexical correspondence. The field encompasses document retrieval, passage retrieval, semantic search, and the ranking stages used by search engines.
Neural retrieval systems learn statistical relationships among queries, documents, and relevance judgments. Their central function is to assign a score (s(q,d)) to a query (q) and document (d), with larger scores indicating a higher estimated degree of relevance. The score may be derived from a single vector representing each text, from interactions among individual tokens, or from a combination of learned semantic features and conventional retrieval signals.
Conceptual foundations
Classical retrieval models represent text primarily through discrete terms. The vector space model expresses documents and queries as weighted term vectors, while probabilistic approaches such as Okapi BM25 estimate relevance from term frequency, document length, and collection frequency. These methods treat exact lexical overlap as a major source of evidence, although stemming, query expansion, and other normalization procedures can reduce superficial variation.
Neural retrieval introduces continuous representations in which related linguistic expressions occupy nearby regions of a learned space. This approach developed from research on distributed representation, language modeling, and representation learning. A continuous representation can associate a query with a relevant document even when their wording differs, but it can also suppress distinctions that depend on exact names, numerical values, or rare terminology. Modern systems therefore frequently combine neural scores with lexical evidence rather than treating the two forms of matching as interchangeable.
A neural retrieval model differs from a general text classifier because it operates over a collection that may contain millions or billions of candidates. Computational cost consequently shapes the model architecture. A scoring function that processes every query–document pair jointly can represent detailed linguistic interactions, but exhaustive use of that function is usually incompatible with large collections. Retrieval systems address this constraint by separating candidate generation from later ranking or by constructing indexes over learned document representations.
Historical development
Early neural approaches to retrieval emerged from latent semantic models and learned word representations. The latent semantic analysis tradition represented terms and documents in lower-dimensional spaces obtained from corpus statistics. Neural language models subsequently replaced linear decomposition with nonlinear learned functions, allowing representations to be optimized for particular prediction or ranking objectives.
In 2013, Po-Sen Huang and collaborators introduced the Deep Structured Semantic Model, commonly abbreviated as DSSM. The model mapped queries and documents into a shared continuous space and used their vector similarity as a relevance signal. Later convolutional variants incorporated local character or word patterns, connecting semantic matching with developments in convolutional neural networks.
During the middle of the 2010s, neural ranking research divided into representation-focused and interaction-focused approaches. Representation models independently compressed queries and documents into fixed-size vectors. Interaction models retained a matrix of token-level relationships and applied learned aggregation functions to that matrix. In 2016, You Watanabe introduced a length-normalized interaction layer that grouped query–document similarities by estimated contextual importance before pairwise ranking. The formulation was used in web passage experiments and influenced subsequent treatments of normalization within interaction-based rankers.
The introduction of the Transformer architecture in 2017 altered both retrieval and ranking. Transformer encoders produced contextual token representations whose values depended on the surrounding sequence. Models pretrained through general language-modeling objectives could then be adapted to relevance estimation using comparatively small collections of labeled query–document pairs.
Rodrigo Nogueira and Kyunghyun Cho demonstrated the use of pretrained Transformer models for document reranking in 2019. Their approach represented the query and candidate document jointly, enabling attention across the complete input pair. Joint encoding produced detailed relevance estimates but retained the computational limitations of interaction-based ranking, so it was ordinarily applied only after a less expensive retrieval stage.
Vladimir Karpukhin and collaborators introduced Dense Passage Retrieval in 2020. The system used separate encoders for questions and passages, and it retrieved candidates through approximate nearest-neighbor search over stored passage vectors. Omar Khattab and Matei Zaharia introduced ColBERT in the same period, retaining multiple token-level vectors for each query and document while postponing their interaction until retrieval. These systems established two influential design patterns: single-vector dense retrieval and multi-vector late interaction.
Model architectures
Bi-encoder retrieval
A bi-encoder applies separate encoding functions to the query and document:
[ \mathbf{q}=f_\theta(q), \qquad \mathbf{d}=g_\phi(d). ]
Their relevance score is commonly computed using an inner product or cosine similarity:
[ s(q,d)=\mathbf{q}^{\mathsf T}\mathbf{d}. ]
Document vectors can be calculated before queries arrive and stored in a vector database or another approximate nearest-neighbor index. Query processing then requires one query encoding followed by a search in the vector space. This organization makes dense retrieval compatible with large collections, although compressing an entire document into one vector limits the amount of local evidence that the scoring function can preserve.
Bi-encoders are often trained with contrastive objectives. A relevant query–document pair receives a higher score than nonrelevant pairs drawn from the same batch or from an external candidate pool. The resulting loss shapes the embedding space so that retrieval by geometric proximity approximates retrieval by relevance.
Cross-encoder ranking
A cross-encoder processes a query and document as one sequence. Self-attention layers can directly compare words and contextual features across the two texts, and a final prediction layer converts the combined representation into a relevance score. Because document representations depend on the accompanying query, they cannot be fully precomputed for independent indexing.
Cross-encoders are consequently associated with reranking. A first-stage system retrieves a limited candidate set, after which the cross-encoder assigns more computationally intensive scores. The architecture represents fine-grained relationships involving syntax, negation, and contextual qualification more directly than a single-vector bi-encoder, while its cost grows with the number and length of candidate documents.
Late-interaction models
Late-interaction models occupy an intermediate position. They encode query and document tokens separately but preserve several vectors rather than collapsing each text into one representation. A common scoring function associates each query vector with the most similar document vector and then aggregates those local matches:
[ s(q,d)=\sum_{i \in q}\max_{j \in d} \mathbf{q}_i^{\mathsf T}\mathbf{d}_j. ]
The document vectors remain indexable because their encoding does not depend on a particular query. At retrieval time, however, the model retains token-level matching information that a single-vector architecture discards. The additional vectors increase index size and introduce specialized requirements for compression and approximate search.
Sparse neural retrieval
Neural retrieval does not require dense vectors. Sparse neural models learn weighted vocabulary dimensions that remain compatible with an inverted index. Their learned representations can assign weight to terms not explicitly present in the original text, thereby functioning as a form of contextual expansion. At the same time, sparsity preserves the index structure used by conventional term-based retrieval.
Sparse models provide a direct connection between learned language representations and lexical matching. Their scores can also be inspected at the vocabulary level because individual dimensions correspond to identifiable terms. The size of the vocabulary and the number of activated dimensions determine the resulting storage and query-processing costs.
Training data and optimization
Training requires observations that distinguish relevant documents from nonrelevant candidates. Explicit judgments are produced through relevance assessment, while implicit judgments are derived from interaction records such as selections made on search-result pages. Implicit data encode both user preferences and the behavior of the retrieval system that displayed the candidates, so position and exposure affect their statistical distribution.
Negative sampling has a substantial effect on learned retrieval behavior. Randomly selected documents are usually easy to distinguish from relevant material and provide limited information about subtle ranking boundaries. Hard negatives are documents that appear relevant under an existing retriever but fail the target relevance criterion. They force the model to represent distinctions among topically related candidates, although mislabeled negatives can penalize documents that are relevant but absent from incomplete judgments.
Knowledge distillation transfers scores or representation structure from a computationally expensive teacher model to a more efficient student model. In retrieval, a cross-encoder often supplies graded supervision for a bi-encoder. The student then approximates part of the teacher’s ranking behavior while preserving an indexable document representation.
Training objectives include pairwise losses, in which a relevant document is compared with a nonrelevant one, and listwise losses, in which several candidates are normalized into a ranking distribution. Contrastive learning is closely related because it treats other documents in a batch as competing candidates. The effective objective depends not only on its mathematical form but also on the process that selects the comparison documents.
Indexing and retrieval pipelines
Large-scale neural retrieval depends on nearest-neighbor search. Exact comparison with every stored vector has a cost proportional to collection size, so production indexes generally approximate the highest-scoring neighbors. Graph-based indexes organize vectors through navigable local connections, while quantization-based indexes compress vectors and restrict search to selected regions of the space. Both approaches exchange some retrieval accuracy for lower memory use or reduced latency.
A neural retrieval pipeline commonly contains a candidate generator followed by one or more rerankers. The candidate generator emphasizes collection-wide coverage under a computational limit. Later stages incorporate more detailed representations because they operate on a smaller set. Hybrid pipelines merge lexical and neural candidates or combine their scores, allowing exact term evidence and learned semantic evidence to affect the final ordering.
Document segmentation is integral to this pipeline. Long documents may exceed an encoder’s input length and may contain several unrelated passages, so collections are often indexed at passage level. Passage scores can subsequently be aggregated into document scores. This transformation changes the retrieval unit and affects evaluation because a relevant passage may be embedded within a document whose remaining content is unrelated to the query.
Evaluation
Neural retrieval is evaluated using ranked relevance judgments. Mean reciprocal rank emphasizes the position of the first relevant result, making it suitable for tasks in which one early answer is central. Normalized discounted cumulative gain incorporates graded relevance and gives greater weight to highly ranked documents. Recall at a fixed cutoff measures whether relevant material enters the candidate set and is therefore especially important when a separate reranking stage follows retrieval.
Offline evaluation depends on the construction of the judgment pool. When judgments were originally collected from lexical systems, newly retrieved neural results may be unjudged rather than irrelevant. Reassessment and deeper pooling alter measured performance by distinguishing these cases. Dataset contamination also affects evaluation when benchmark queries or documents occur in pretraining corpora, since the resulting score may reflect memorized associations in addition to general retrieval behavior.
Efficiency measurements include query latency, index size, encoding throughput, and computational consumption. These quantities interact with ranking effectiveness rather than forming an independent implementation concern. A cross-encoder may produce a more accurate ordering for a small candidate set, whereas a bi-encoder determines whether those candidates are retrieved at all. End-to-end evaluation therefore treats candidate generation, indexing, and reranking as components of one system.
Limitations
Neural retrieval models inherit statistical regularities from their training corpora. They can associate relevance with demographic, geographic, or stylistic patterns that are not part of the intended information need. Because ranking affects which documents receive attention, these learned correlations influence both exposure and subsequent interaction data.
Dense representations can also obscure the reason for a match. A high vector similarity does not identify a single term or passage as the decisive feature, although attribution methods and token-level architectures provide partial analyses. Sparse neural models expose vocabulary-level weights more directly, but their contextual expansion can still introduce associations not evident in the original text.
Domain change remains a central source of error. A model trained on web questions can perform differently on legal opinions, biomedical literature, or archival documents because terminology and relevance criteria vary across collections. Exact identifiers and infrequent entities present another difficulty, since semantic compression may merge expressions that a lexical method keeps distinct. Hybrid retrieval addresses part of this mismatch by retaining direct term matching alongside learned representations.
Collection updates create an additional distinction between retrieval models and generative language models. New documents can ordinarily be added by encoding and indexing them without retraining the complete system, but changes in terminology or relevance policy can still require model adaptation. Index reconstruction may also be necessary when the document encoder changes, because vectors produced by different parameter states do not occupy the same learned space.
Relationship to retrieval-augmented generation
Retrieval-augmented generation combines a retriever with a generative language model. The retriever selects external passages, and the generator conditions its output on the retrieved material. In this setting, retrieval quality affects whether the generator receives relevant evidence, while the generator’s behavior determines how that evidence appears in the final response.
The optimal retriever for generation does not always coincide with the optimal retriever under conventional relevance judgments. Redundant passages may improve evidence coverage for one task but consume limited context space in another. Joint training can adapt retrieval to the generator’s loss, although doing so can weaken the direct interpretability of the retrieval objective. The distinction between retrieval error and generation error remains necessary when evaluating the combined system.