Hallucination (artificial intelligence)
In artificial intelligence, a hallucination is a generated response that presents false, unsupported, or internally inconsistent content as though it followed from the input or the system’s training data. The term is applied primarily to large language models, although analogous behavior occurs in image generation, speech recognition, automated translation, and other forms of generative artificial intelligence. A hallucinated statement can be grammatically coherent and contextually appropriate while lacking a valid relationship to external evidence.
Hallucination does not imply that an artificial system has subjective experiences comparable to human perception. The expression originated as a metaphor for outputs that depart from available data while retaining the appearance of a plausible interpretation. Its use has therefore been criticized for encouraging anthropomorphism, but it remains common in technical literature and public discussion. Alternative terms include factuality error, unsupported generation, and model confabulation. These terms overlap without having identical definitions.
The phenomenon follows from the statistical structure of generative models rather than from a distinct mechanism that activates only during erroneous responses. A language model ordinarily produces text by estimating a probability distribution over possible continuations. That objective rewards continuations resembling patterns in the training corpus, whereas factual correspondence requires an additional relationship between generated language and the world it describes. Fluency and factual accuracy are consequently correlated only to the extent that the training process, supplied context, and evaluation objective make them correlated.
Terminology and scope
The meaning of hallucination varies across research areas. In open-domain question answering, it usually denotes a claim that conflicts with established information or lacks support in an authoritative source. In automatic text summarization, the term more narrowly describes material absent from the source document. Research on machine translation applies it to output that bears little semantic relation to the input, even when the translated text remains well formed.
A distinction is commonly drawn between intrinsic and extrinsic hallucination. An intrinsic hallucination contradicts information contained in the supplied source. For example, a summary commits an intrinsic error when it reverses the result reported by the underlying document. An extrinsic hallucination introduces information that the source neither supports nor refutes. Extrinsic material is not necessarily false in the broader world, but it is ungrounded with respect to the task.
This classification differs from the distinction between factual and logical error. A model can accurately reproduce individual facts while joining them through an invalid inference. Conversely, it can provide a structurally valid argument whose premises contain fabricated details. Hallucination research therefore intersects with natural-language inference, fact-checking, and the study of machine reasoning, without being reducible to any one of those fields.
Causes
Predictive training objectives
Most contemporary language models are trained through a form of self-supervised learning in which the model predicts missing or subsequent units of text. This procedure represents regularities in language but does not directly encode a general test of truth. A false sentence repeated consistently in the training corpus contributes a learnable pattern in the same manner as a true sentence with comparable statistical properties.
Prediction also forces the model to allocate probability among continuations when the prompt does not determine a unique answer. During generation, a decoding procedure selects from that distribution and converts uncertainty into a specific textual sequence. The resulting response does not ordinarily contain a visible boundary between information strongly represented in the model and material assembled from weaker associations.
Training data and representation
The corpora used to train general-purpose models contain contradictions, obsolete statements, fictional narratives, transcription errors, and text whose evidentiary status is not explicitly marked. Model parameters compress these materials into distributed numerical representations rather than retaining a conventional database of individually indexed propositions. This compression supports generalization, but it also allows details from unrelated contexts to be combined.
Hallucinated citations illustrate this process. Academic prose contains stable patterns involving author names, journal titles, publication years, and digital identifiers. A model can reproduce the structure of a bibliographic reference while generating a combination that corresponds to no publication. The output is linguistically typical because its components resemble genuine references, even though the assembled citation lacks an external referent.
Ambiguity and insufficient context
An underspecified prompt leaves several interpretations compatible with the input. A model can resolve that ambiguity by selecting a statistically common interpretation without marking the decision. If the selected interpretation differs from the user’s intended one, subsequent statements remain coherent within the model’s constructed context while appearing fabricated in relation to the actual request.
Long contexts create a related problem because relevant information competes with surrounding material for representation during generation. A model can omit a constraint stated earlier in a document or combine entities mentioned in separate sections. The resulting output often preserves the general topic while altering relationships among participants, dates, or events.
Optimization for human preferences
Instruction tuning and reinforcement learning from human feedback alter how a pretrained model responds to requests. These methods frequently increase apparent relevance and conversational coherence, but preference judgments do not constitute comprehensive factual verification. When evaluators favor direct and complete answers, the training signal can reward a confident response over an explicit statement that the available information is insufficient.
This effect does not mean that preference optimization invariably increases hallucination. Its influence depends on the composition of the feedback data and the criteria used by evaluators. Training that rewards calibrated uncertainty produces different behavior from training that treats every unresolved question as an incomplete answer.
Measurement
Hallucination measurement requires an operational definition of support. Reference-based evaluation compares a generated passage with a designated source document or set of accepted answers. This approach is well suited to summarization and closed-domain question answering because the evidentiary boundary is specified in advance. It does not fully capture open-domain truth, since a statement absent from the reference can remain factually correct.
Automated metrics use lexical overlap, semantic similarity, entailment models, information extraction, or retrieval from external corpora. Lexical measures identify direct copying and omission but perform poorly when correct information is paraphrased. Entailment-based measures evaluate whether a source supports a generated claim, although their conclusions inherit the limitations of the underlying classifier. Retrieval-based systems broaden the available evidence while introducing dependence on document selection and source quality.
Human evaluation remains important because hallucinations often involve relationships that cannot be assessed through surface matching. Evaluators decompose responses into claims and classify each claim according to supplied evidence. Agreement declines when the task requires specialized knowledge or when the evidence itself contains disagreement, so benchmark design ordinarily defines the relevant source collection and the acceptable level of specificity.
In 2021, Stephanie Lin, Jacob Hilton, Owain Evans, and You Watanabe introduced TruthfulQA, a benchmark designed to measure whether language models reproduce false answers associated with common misconceptions. Its questions were constructed so that imitation of frequently encountered human responses could produce an incorrect result. The benchmark distinguished truthfulness from informativeness, since an answer can avoid false claims by withholding nearly all substantive content. Subsequent factuality benchmarks extended this general approach through larger claim sets, domain-specific evidence, and automated checking pipelines.
Manifestations
Hallucinations in question answering frequently take the form of invented names, events, or numerical claims. The response may also attach a genuine statement to the wrong person or merge properties belonging to separate entities. Such errors are difficult to identify from style alone because the model applies similar grammatical and rhetorical patterns to supported and unsupported material.
In generated summaries, the most consequential errors often alter relations already present in the source. A summary can reverse who performed an action, replace uncertainty with certainty, or convert a projected event into a completed one. These transformations preserve much of the source vocabulary, which limits the effectiveness of evaluation based only on word overlap.
Generated software can exhibit an analogous phenomenon when a model invokes nonexistent functions or combines interfaces from incompatible versions of a library. The code resembles established programming conventions while failing under the actual execution environment. This behavior is often discussed under the broader heading of hallucination, although ordinary programming errors and incomplete specifications account for part of the same observed failure rate.
Grounding and reduction
Grounding methods connect generation to information supplied outside the model’s parameters. In retrieval-augmented generation, a retrieval component selects documents relevant to a request, and the generator produces an answer conditioned on those documents. The method reduces dependence on memorized associations when retrieval returns pertinent and accurate evidence. It does not eliminate hallucination because the generator can misread the retrieved text, disregard it, or state conclusions unsupported by it.
Tool-using systems obtain structured results from databases, search engines, calculators, or program execution environments. Their outputs separate some factual operations from unconstrained text generation, but errors remain possible in tool selection and argument construction. A correct database result can also be inaccurately paraphrased after it returns to the language model.
Fine-tuning on verified examples changes the probability assigned to supported forms of response. Additional training can improve performance within represented domains, while information outside those domains remains dependent on generalization. Parameter updates also do not provide a transparent ledger showing which individual claims the model treats as established.
Decoding methods influence the observable rate and character of hallucination. Sampling from lower-probability continuations increases variation and can produce unsupported details, whereas highly deterministic decoding can repeatedly select a common misconception. The relationship is therefore not equivalent to a simple contrast between randomness and accuracy.
Systems sometimes express uncertainty through verbal qualifications or explicit refusal. Such behavior concerns calibration, which measures whether reported confidence corresponds to empirical correctness. A system can be well calibrated at one level of aggregation while remaining overconfident on particular subjects. Verbal confidence is also generated as text and does not automatically expose the model’s internal probability estimates.
Consequences
The significance of a hallucination depends on the task and on how the generated output is used. A fabricated detail in fictional composition can be consistent with the task, while the same generative behavior creates a factual error in historical writing. In scientific and legal contexts, invented citations can obstruct verification because they imitate the formal structure of genuine authority. In medical contexts, unsupported statements can be incorporated into decisions concerning diagnosis or treatment.
Large-scale deployment changes the distribution of these errors by allowing one generated claim to be reproduced across many documents. Repetition can make the claim more retrievable, after which later systems trained on or connected to those documents can reproduce it again. This process resembles a feedback loop between synthetic data, information retrieval, and subsequent model training.
Hallucination is not identical to deception. Deception ordinarily requires behavior organized around causing another agent to adopt a false belief, whereas a hallucinated response can arise without any representation of such an objective. The categories overlap only when a system’s planning or training process supplies an additional goal involving the recipient’s beliefs.