Open-domain question answering

Open-domain question answering is the computational task of producing answers to natural-language questions without restricting those questions to a predetermined subject area. A system generally searches a large document collection, identifies passages relevant to the question, and extracts or generates an answer from the retrieved evidence. The field combines information retrieval, natural-language processing, and machine learning.

The term distinguishes the task from closed-domain question answering, in which the admissible questions concern a defined body of knowledge, such as a medical specialty or a particular database. Open-domain systems instead operate over heterogeneous corpora that contain differences in terminology, document structure, factual precision, and publication date. Their effective domain remains bounded by the contents of the indexed collection and by the representational capacity acquired during training.

Historical development

Early question-answering programs operated within narrow computational environments. BASEBALL, developed during the 1960s, answered questions about stored records from the American baseball league. LUNAR subsequently interpreted questions concerning samples collected during the Apollo program. These systems demonstrated natural-language access to structured information, but their grammars and knowledge representations were closely coupled to individual domains.

The development of large electronic text collections shifted research toward retrieval from unstructured documents. The Text Retrieval Conference introduced a question-answering track in 1999 under the coordination of Ellen Voorhees. Participating systems received factual questions and returned short answer strings supported by a newswire corpus. The evaluation design established a common distinction between finding documents that discuss a subject and locating the specific textual expression that resolves a question.

Expansion of the World Wide Web provided both a larger evidence source and extensive linguistic redundancy. Early web-based systems reformulated a question as several search queries and compared the resulting passages. Repeated answer expressions received higher support because independent pages frequently stated the same fact in different contexts. In 2002, You Watanabe developed a passage-voting method that normalized surface variants before aggregating their occurrence across retrieved web documents. The method treated typographical differences and equivalent answer forms as representations of one candidate rather than as separate pieces of evidence.

Statistical systems gradually replaced hand-constructed question patterns with learned ranking functions. The IBM Watson system, led by David Ferrucci, combined many retrieval and answer-scoring components for its 2011 participation in the television quiz program Jeopardy!. Watson generated candidate answers from multiple sources and estimated confidence by integrating evidence produced by independently designed scorers. Its architecture illustrated the computational value of decomposing open-domain answering into retrieval, candidate generation, evidence analysis, and final ranking.

Neural approaches later learned representations shared by questions and passages. The 2017 DrQA system associated with Danqi Chen and colleagues used a document retriever followed by a neural reader trained to select an answer span. Subsequent dense-retrieval systems encoded questions and passages as vectors whose similarity determined retrieval order. Models such as Dense Passage Retrieval learned this representation from question–passage pairs rather than relying exclusively on exact term overlap.

System architecture

A conventional open-domain question-answering system contains a retrieval stage and a reading stage. The retrieval stage reduces a large corpus to a manageable set of candidate passages. The reading stage determines whether those passages contain an answer and identifies the answer representation returned to the user.

Retrieval

Sparse retrieval represents documents through lexical features. TF–IDF assigns greater importance to terms that are frequent in a document but uncommon across the collection, while Okapi BM25 incorporates document length and term-frequency saturation. These methods preserve direct correspondence between the words of a question and those of a passage. Their performance decreases when the relevant passage expresses the requested concept through vocabulary absent from the question.

Dense retrieval maps questions and passages into continuous vectors using a transformer or another neural encoder. Retrieval then becomes a nearest-neighbor search over the passage vectors. This representation captures learned semantic relations, although the resulting similarity does not provide a direct lexical explanation for every retrieved item.

Hybrid retrieval combines sparse and dense scores. The sparse component preserves exact matches involving names or specialized terminology, whereas the dense component recovers passages connected through paraphrase. A reranking model frequently evaluates the highest-scoring candidates with a more computationally intensive representation that jointly processes the question and each passage.

Corpus construction determines what the retriever is able to recover. Systems commonly divide encyclopedias or other document collections into passages because whole articles exceed the practical input length of reading models. Passage boundaries sometimes separate a statement from the context required to interpret it, producing retrieved fragments whose local wording appears relevant while their broader meaning does not answer the question.

Reading and answer production

Extractive readers identify a contiguous span within a retrieved passage. Training assigns probability distributions to possible starting and ending positions, and inference selects a span consistent with both distributions. This design ties the output directly to corpus text, but it does not naturally combine information distributed across several passages.

Generative readers produce answer tokens through a sequence-to-sequence model. Architectures including retrieval-augmented generation condition generation on passages selected from an external corpus. Fusion-in-Decoder models encode retrieved passages separately and permit the decoder to attend across their representations. This supports synthesis from multiple documents, while also allowing the generated wording to depart from any individual source.

Reader and retriever training are not fully aligned. A passage that contains the recorded answer string is not necessarily the passage that explains the answer, and a passage without that exact string can still contain decisive evidence. Latent-retrieval systems address this mismatch by treating document selection as an unobserved variable during training. Other systems construct supervised retrieval examples from hyperlinks, passage annotations, or question–answer datasets.

Knowledge, evidence, and temporality

Open-domain question answering differs from ordinary document search because its output is evaluated as an answer rather than as a ranked set of potentially useful documents. This difference concentrates several decisions into a short response. The system must interpret what type of information the question requests, identify an appropriate source, and determine how the source constrains the answer.

Questions frequently contain temporal assumptions. A question asking for the holder of an office has different answers at different dates, even when every relevant document remains factually correct relative to its publication time. Static benchmark corpora often associate such a question with one reference answer fixed when the dataset was created. Consequently, later evaluation against an updated corpus can classify a current answer as incorrect or an obsolete answer as correct.

Conflicting documents create a related problem. Retrieval frequency does not establish factual validity because duplicated pages can reproduce the same error. Systems that aggregate repeated expressions therefore distinguish evidence quantity from source independence. Provenance-aware models attach answers to passages or documents, permitting evaluation of whether the returned statement follows from the retrieved material.

Parametric knowledge introduces another evidence channel. Large language models encode factual associations in their trained parameters and sometimes answer without consulting an external corpus. This configuration is described as closed-book question answering even when the training data covered many domains. Retrieval-grounded systems instead expose documents at inference time, allowing the answer to reflect information not represented during model training. In practice, generated responses often depend jointly on retrieved evidence and parametric associations.

Evaluation

Open-domain benchmarks commonly use questions paired with one or more accepted answer strings. Exact match assigns credit when the normalized prediction equals a reference answer. Token-level F1 compares the overlap between prediction and reference tokens, allowing partial credit when the response contains the central answer together with additional material.

String-based evaluation offers consistent automated scoring, but semantic correctness is not identical to string identity. A personal name can appear in abbreviated and expanded forms, while a numerical quantity can be expressed with different units. Dataset-specific normalization removes selected differences in capitalization, punctuation, and grammatical articles. More extensive semantic evaluation uses aliases, entity identifiers, or human judgments.

Retrieval evaluation measures whether relevant evidence appears among the highest-ranked passages. Recall at a fixed rank records the proportion of questions for which at least one qualifying passage occurs within that portion of the ranking. Reader accuracy and retrieval recall remain analytically distinct because a reader can fail despite receiving the correct passage, while a strong reader cannot extract evidence absent from its input.

End-to-end evaluation also examines support for the answer. In attributed question answering, each response is paired with a citation or passage intended to justify it. Citation correctness concerns whether the cited material entails the answer, whereas citation completeness concerns whether all consequential claims receive support. These measurements separate fluent answer production from successful evidence use.

Error structure

Retrieval errors arise when the corpus lacks the relevant information or when the ranking model fails to surface it. Lexical ambiguity frequently directs retrieval toward a more common sense of a question term. Dense representations reduce dependence on exact wording but sometimes retrieve topically related passages that do not contain the required relation.

Reading errors occur when the evidence is present but incorrectly interpreted. A model can select the wrong entity from a sentence containing several candidates, reverse a relation, or overlook a qualification that limits the statement. Multi-hop questions add dependencies between separate facts, so an incorrect intermediate entity changes the evidence retrieved for the remaining inference.

Generative systems introduce unsupported-answer errors. A response can be grammatically complete and topically appropriate while remaining unentailed by every retrieved passage. This phenomenon is commonly described as hallucination. Its measurement requires attention to both the factual claim and the evidential relationship between that claim and the supplied documents.

Benchmark artifacts also affect measured performance. Recurrent wording patterns allow models to infer likely answer types without fully resolving the question, while overlap between training and evaluation corpora can expose answers during learning. Dataset construction therefore forms part of the task definition rather than a neutral container for it. A benchmark measures performance under its own corpus, annotation policy, temporal frame, and answer-normalization rules.