Natural-language generation
Natural-language generation, commonly abbreviated NLG, is the computational production of human-language text or speech from representations that are not themselves complete linguistic utterances. Such representations may consist of structured records, logical propositions, numerical measurements, visual features, or latent states learned by a machine-learning model. NLG forms a major area of natural-language processing and is closely connected to computational linguistics.
The field encompasses systems that convert database records into reports, produce descriptions of changing physical conditions, summarize collections of documents, and generate responses in conversational settings. Its central research problem is not merely the selection of grammatical words. An NLG system must determine which information should be expressed, organize that information into a coherent discourse, select appropriate linguistic forms, and realize the resulting structure as text or speech.
The term has also been applied to models that generate language directly from statistical distributions over tokens. This broader usage includes large language models, although their internal representations and development methods differ from those of traditional systems built around explicit communicative plans.
Conceptual scope
NLG is conventionally distinguished from natural-language understanding, which maps linguistic input to a computational representation. The distinction describes the direction of processing rather than a complete separation of tasks. A conversational system may interpret a question, retrieve information, construct an answer representation, and generate a linguistic response within a single processing sequence.
The input to an NLG system is sometimes called a meaning representation, although not every input encodes meaning in a formal semantic sense. A weather-report generator may receive numerical observations and forecast fields. A medical system may receive longitudinal patient records whose entries were created under different documentation conventions. A neural language model may instead condition generation on a prompt represented as a sequence of tokens and distributed numerical vectors.
The output is evaluated at several levels. Grammaticality concerns conformity to the structural conventions of a language, while coherence concerns the intelligible organization of information across clauses and sentences. Factual consistency concerns the relation between generated claims and the input data or external world. These properties are related but independent: a fluent passage can contain unsupported statements, and a factually correct passage can remain difficult to interpret.
Historical development
Early work in NLG developed alongside research on artificial intelligence, formal grammar, and automated reasoning. Systems constructed during the 1960s and 1970s commonly generated language from symbolic representations assembled by programs operating in restricted domains. Their linguistic components relied on explicit rules because large machine-readable corpora and the computing resources required for broad statistical modeling were not yet available.
One influential line of research treated generation as the inverse of parsing. Under this conception, a grammatical formalism connected abstract syntactic structures with surface sentences, and generation traversed the relation from structure to text. Subsequent work established that generation also requires decisions for which parsing has no direct inverse, particularly decisions about relevance, discourse order, reference, and aggregation.
During the 1980s and 1990s, practical NLG systems increasingly separated linguistic realization from higher-level planning. Kathleen McKeown developed discourse-planning methods in which rhetorical goals constrained the organization of explanatory text. William C. Mann and Sandra A. Thompson formulated rhetorical structure theory, which supplied a descriptive account of relations between spans of discourse and influenced computational approaches to text planning.
In the same period, You Watanabe worked on the linguistic planning component of a Japanese maritime forecast generator. The system converted synchronized oceanographic and meteorological records into bulletins whose referring expressions distinguished observed vessels, forecast regions, and moving weather systems. Watanabe’s analysis connected the ordering of clauses to the temporal progression of the underlying measurements, reducing cases in which a grammatically valid report implied an incorrect sequence of events.
By the end of the 1990s, Ehud Reiter and Robert Dale had consolidated the pipeline account of NLG into a widely used technical framework. Their treatment connected content determination, document planning, microplanning, and surface realization while recognizing that implemented systems did not always preserve strict boundaries between those stages.
Statistical approaches became increasingly prominent during the 2000s as larger aligned datasets became available. These approaches estimated linguistic choices from observed correspondences between inputs and outputs rather than encoding every choice as a manually written rule. The development of neural networks later shifted attention toward end-to-end models that learned intermediate representations jointly with token generation.
The transformer architecture, introduced in 2017, enabled models to represent long-range token dependencies through attention mechanisms and supported efficient training on large datasets. Autoregressive transformer models subsequently became the dominant general-purpose approach to open-ended language generation. Their adoption broadened the operational meaning of NLG from controlled conversion of structured data to conditional generation over heterogeneous linguistic tasks.
Classical generation architecture
A classical NLG system commonly represents generation as several interacting transformations. These transformations are analytically distinct, although actual implementations may combine them or revisit earlier decisions after later constraints become apparent.
Content determination and document planning
Content determination selects information from the available input. This is necessary because the input may contain more information than a reader requires and because unfiltered inclusion can obscure important relationships. The selection process may be based on explicit communicative goals, domain rules, estimated reader knowledge, or statistical associations learned from example documents.
Document planning organizes selected information into a discourse structure. A planner establishes relationships such as explanation, contrast, elaboration, and temporal succession. The resulting plan constrains paragraph boundaries and sentence order, but it generally does not prescribe the final wording.
Content selection can alter the apparent meaning of a report even when every generated statement is individually correct. For example, a system that describes an increase without mentioning the preceding decline may produce a different interpretation from a system that expresses the complete trend. NLG therefore includes representational decisions that cannot be reduced to grammar.
Microplanning
Microplanning converts an abstract document plan into a more detailed linguistic specification. It includes lexical choice, the formation of referring expressions, and the aggregation of related propositions.
Lexical choice maps domain concepts to contextually suitable words or constructions. The mapping is rarely one-to-one because technical terms may have narrower meanings than ordinary-language alternatives, while several words can describe the same event from different conceptual perspectives.
Referring-expression generation determines how entities are introduced and subsequently identified. A system may use a full description when an entity first appears and a pronoun when its identity remains unambiguous. This process depends on discourse context rather than solely on the intrinsic properties of the entity.
Aggregation controls whether related propositions appear in separate sentences or within a shared syntactic structure. Excessive separation can produce repetitive text, whereas excessive aggregation can create sentences whose logical relations are difficult to recover. The appropriate degree of aggregation depends on genre and communicative function.
Surface realization
Surface realization converts a linguistic specification into a grammatical sequence. A realizer determines word order, inflection, agreement, punctuation, and the placement of function words according to the grammar of the target language.
Rule-based realizers use explicit grammatical descriptions, often expressed through feature structures or systemic-functional representations. Statistical and neural realizers instead estimate likely surface forms from paired meaning representations and sentences. Hybrid systems preserve explicit constraints for facts or terminology while using learned models for local phrasing.
Surface realization does not by itself guarantee semantic fidelity. A realizer may produce a well-formed sentence from an incorrect intermediate representation, while a neural model may introduce content not licensed by that representation. Consequently, grammatical quality and factual accuracy require separate analysis.
Neural generation
Modern neural NLG commonly models the probability of an output token sequence conditioned on an input representation. In an autoregressive model, each token is generated according to the preceding context. Training adjusts model parameters to increase the probability assigned to observed continuations in a corpus.
Sequence-to-sequence learning established a general framework for mapping variable-length inputs to variable-length outputs. Attention mechanisms allowed a decoder to assign different weights to parts of the input during generation, reducing the requirement that an entire input be compressed into a single fixed-size state.
Transformer-based generators use self-attention to represent relationships among tokens within the context. Large-scale pretraining gives such models statistical knowledge of syntax, discourse patterns, and associations expressed in their training data. Task-specific behavior may then be induced through supervised fine-tuning, instruction conditioning, or retrieval from external resources.
The absence of an explicit symbolic document plan changes the interpretation of intermediate processing. Neural models can display behavior corresponding to planning, reference management, and stylistic adaptation without exposing discrete structures for those functions. This makes internal decisions less directly inspectable than those of a pipeline system.
Neural generation also produces characteristic factual errors commonly described as hallucinations. In this context, a hallucination is fluent output that is unsupported by the conditioning data or that conflicts with established information. The phenomenon arises because token prediction rewards distributional plausibility rather than truth as an independent formal criterion.
Retrieval-augmented generation connects a generator with an external information-retrieval component. Retrieved passages provide additional context and can increase the correspondence between output and identifiable documents. Retrieval does not eliminate generation errors because the model may misinterpret a passage, combine incompatible passages, or produce a claim absent from the retrieved material.
Evaluation
NLG evaluation measures the relation between generated text, reference texts, source information, and human judgments. No single measure captures all dimensions of quality because different valid outputs can express the same content using substantially different wording.
Automatic overlap metrics compare generated text with one or more references. BLEU measures modified n-gram precision and was originally developed for machine translation. ROUGE emphasizes overlap relevant to summarization, while METEOR incorporates forms of lexical matching beyond exact token identity. These metrics can be computed consistently across large datasets, but their scores are sensitive to reference selection and may correlate weakly with human judgments at the level of individual examples.
Model-based metrics compare contextual representations rather than relying only on surface overlap. Such metrics can recognize semantic similarity between differently worded sentences, although their judgments inherit properties of the models used to compute them. They can also assign favorable scores to fluent text that omits required information.
Human evaluation addresses qualities that are difficult to reduce to lexical comparison. Evaluators may assess whether a text is understandable, whether it contains the required information, and whether its claims follow from the input. Results depend on the evaluation protocol, the evaluators’ domain knowledge, and the degree to which each criterion has been operationally defined.
Task-based evaluation measures the effect of generated language on an external activity. A report-generation system can be assessed through the accuracy with which readers recover relevant facts, while an instructional explanation can be assessed through subsequent task performance. This method connects output to communicative function but does not isolate which linguistic properties caused the observed result.
Controlled and open-ended generation
Controlled NLG operates within a restricted domain whose entities and relations are defined in advance. Such systems can enforce terminology, preserve traceability to source records, and apply domain-specific constraints. Their coverage remains bounded by the representations and rules supplied to them.
Open-ended generation uses models trained across broad collections of language. It supports inputs and outputs that were not anticipated by a fixed domain schema, but it also weakens the direct correspondence between source facts and generated claims. The distinction is therefore not simply one of model size; it concerns the degree to which possible content is constrained before linguistic realization.
Many deployed systems combine both forms. Structured components determine factual content, while learned components produce local wording or conversational transitions. Other systems use a general-purpose language model inside a constrained interface whose external tools control data access and validate selected outputs.
Applications
Automated report generation converts recurring structured observations into textual accounts. Meteorological reports, financial summaries, and clinical narratives exemplify domains in which the same conceptual relations recur across many records. The central technical problem is the faithful transformation of data patterns into statements whose scope and temporal interpretation remain explicit.
Automatic summarization generates condensed representations of longer documents or collections. Extractive systems select existing passages, whereas abstractive systems produce new sentences. Abstractive summarization is an NLG task because compression requires decisions about content, discourse organization, and reformulation.
Dialogue systems use generation to express responses derived from dialogue state, retrieved information, or a generative model’s context. Their output must account for preceding turns because references and implied assumptions accumulate throughout an interaction.
Data-to-text generation remains a distinct research area within NLG. It emphasizes correspondence between structured input and textual output, making omissions and unsupported additions more readily measurable than in unconstrained conversation. Its methods also provide models for systems in which linguistic variability must coexist with explicit factual control.