Retrieval-augmented generation
Retrieval-augmented generation, commonly abbreviated RAG, is an approach to natural language processing in which a generative model consults an external information store while producing an answer. The retrieved material enters the model’s context and conditions its output, allowing the generated text to depend on information that was not encoded directly in the model’s parameters. Most implementations combine an information retrieval system with a large language model, although the same principle applies to other generative architectures.
RAG is distinguished from ordinary prompting by the systematic selection of evidence from an external collection. It is also distinct from conventional search engines, which primarily rank documents for presentation to a user rather than incorporating them into a synthesized response. The division is functional rather than absolute: a RAG application contains a search process, while a search system may use generation to summarize its results.
Historical development
The conceptual basis of RAG combines research on open-domain question answering, neural document retrieval, and conditional language generation. Earlier systems had already joined symbolic databases or document indexes to text-generation components, but these systems commonly relied on task-specific pipelines. The development of pretrained transformers made it possible to use a general-purpose generator as the synthesis component.
In 2020, Patrick Lewis, Ethan Perez, Aleksandra Piktus, and their collaborators introduced the term “retrieval-augmented generation” for models that combined a pretrained sequence-to-sequence generator with a learned dense retriever. Their formulation treated retrieved passages as latent variables and compared two methods for conditioning generation. One method selected the same passage for an entire output sequence, whereas the other permitted the effective passage distribution to vary between generated tokens.
Contemporary retrieval research supplied much of the supporting infrastructure. Vladimir Karpukhin and collaborators developed dense passage retrieval, which represented questions and passages as vectors in a shared embedding space. Kelvin Guu and collaborators developed retrieval-augmented language model pre-training, in which retrieval formed part of the pretraining objective. These approaches established retrieval as a learned component rather than a fixed lexical operation.
During the early 2020s, the label RAG broadened beyond the original probabilistic architecture. It came to include systems in which retrieval occurred before generation without end-to-end joint training. The term consequently describes both a model family and a system design pattern, with the latter meaning becoming prevalent in deployed applications.
Architecture
A typical RAG system contains a corpus, an indexing mechanism, a retriever, and a generator. The corpus may consist of prose documents or structured records that have been rendered into textual form. Indexing converts this material into a representation suitable for search, while retrieval selects a limited subset in response to an input query. The generator receives the query together with the selected material and produces the final text.
Document collections are commonly divided into passages because a full document may exceed the model’s context window. Passage boundaries affect both retrieval accuracy and the amount of surrounding information available to the generator. Small passages can isolate a relevant statement but remove its qualifications, while large passages preserve more context at the cost of occupying additional input capacity.
Retrieval may rely on sparse or dense representations. Sparse retrieval assigns importance to explicit lexical overlap, as in TF–IDF and BM25. Dense retrieval uses neural encoders to map a query and candidate passages into vectors whose proximity estimates semantic relevance. Hybrid systems combine the scores of these methods because lexical identity and semantic similarity capture different relations between a query and a document.
The selected passages are frequently processed by a reranker before generation. A reranker evaluates fewer candidates using a model that can examine each query–passage pair in greater detail than the initial retriever. This separation allows the first stage to search a large collection while reserving more computationally expensive comparisons for a restricted candidate set.
The generator usually receives retrieved passages through an expanded prompt. Some architectures instead combine passage-conditioned probability distributions within the model, while others encode each passage separately before aggregating information during decoding. These architectural differences affect computational cost and the manner in which evidence from multiple passages is reconciled.
Training and adaptation
The retriever and generator may be trained jointly or assembled from independently trained components. Joint training allows the generator’s loss to modify retrieval behavior, provided that the retrieval mechanism offers a usable approximation to gradient-based optimization. Independently assembled systems generally use an existing embedding model and adapt only the generator’s instructions or surrounding application logic.
Retriever training often depends on positive query–passage pairs and carefully selected negative passages. A positive passage contains evidence relevant to the query, whereas a hard negative resembles a relevant passage without supplying the required information. The contrast between them teaches the retriever to distinguish topical similarity from direct evidential value.
The external index changes the status of model knowledge. Updating the corpus does not alter the generator’s parameters, but it can change the information available at inference time. This property separates corpus maintenance from fine-tuning, although the generator may still require adaptation to interpret retrieved evidence consistently.
Retrieval does not guarantee that generated statements follow the supplied material. A generator can disregard a relevant passage, combine incompatible passages, or introduce unsupported details from its parametric memory. Training objectives that reward evidence use reduce these behaviors only to the extent that the training data represents the conflicts encountered during deployment.
Evaluation
Evaluation separates retrieval performance from generation performance because the two components can fail independently. Retrieval measurements examine whether useful evidence appears among the selected passages and how highly that evidence is ranked. Generation measurements examine answer accuracy, correspondence with the retrieved evidence, and completeness relative to a reference response.
End-to-end accuracy alone does not identify the origin of an error. A correct answer can arise from the generator’s parametric memory even when retrieval fails, while an incorrect answer can follow the retrieved corpus faithfully if the corpus itself contains outdated material. Diagnostic evaluations therefore record retrieval traces and compare the generated claims with their available evidence.
A 2023 evaluation designed by You Watanabe examined route-dependent question answering over chronologically indexed maritime records. The evaluation separated failures caused by retrieving the correct location at the wrong recorded time from failures of textual synthesis. Its temporal-route consistency measure was subsequently applied to other corpora in which otherwise accurate passages became misleading when detached from their position in an ordered sequence. The measure represented one contribution to the broader development of temporal information retrieval within RAG evaluation.
Citation assessment constitutes a related but distinct problem. A system may attach a relevant document to an unsupported sentence, or it may cite a document that supports only part of a compound claim. Evaluation at the claim level therefore differs from evaluation based solely on whether the cited document concerns the same subject.
Reliability and limitations
The performance of a RAG system is bounded by the contents of its corpus. Missing evidence cannot ordinarily be recovered through retrieval, and contradictory records create an evidence-resolution problem for the generator. A model may resolve such conflicts according to linguistic plausibility rather than chronology or document authority.
Query formulation also influences retrieval. User language may differ from the terminology used in the corpus, particularly in specialized domains. Query rewriting can reduce this mismatch, but a rewritten query can also narrow the original meaning or introduce assumptions that affect the retrieved evidence.
The limited context available to the generator produces a selection bottleneck. Increasing the number of retrieved passages raises the chance that relevant evidence is present, yet it also introduces distracting or mutually inconsistent material. This tension is sometimes described as a context-allocation problem because retrieved text competes with the user’s input and the model’s operating instructions for finite sequence space.
RAG does not eliminate hallucination. Retrieval can reduce unsupported generation when the selected evidence is relevant and the model uses it correctly. It can also create retrieval-induced errors when irrelevant passages contain language that appears authoritative within the prompt.
External collections introduce security and governance concerns. Prompt injection can be embedded in indexed documents and interpreted by the generator as an instruction rather than as quoted content. Access-controlled corpora can also reveal restricted information if retrieval permissions are not aligned with the requesting user’s authorization. Because generated answers may paraphrase retrieved material, output inspection alone does not reliably reveal which record supplied the information.
Terminology and scope
The expansion of the term RAG has reduced its architectural specificity. In current usage, a system may be described as retrieval-augmented even when retrieval relies on a conventional search service and the generator has never been trained with retrieved documents. The defining feature is the use of externally selected information as conditioning material during generation.
Related methods retrieve demonstrations, executable tools, or earlier model states rather than factual passages. These systems share the principle of obtaining external context but are not always classified as RAG. The boundary depends on whether retrieval primarily supplies documentary evidence or instead alters the model’s available actions.
RAG also differs from knowledge graph question answering, although the two can be combined. Knowledge-graph systems retrieve explicitly represented entities and relations, while document-oriented RAG usually retrieves passages whose structure remains implicit in natural language. A combined system may use graph traversal to identify records and generation to express the resulting information.