Natural language generation

Natural language generation (NLG) is the computational production of human language from structured or unstructured representations of information. It forms a subfield of artificial intelligence and computational linguistics, and it is commonly treated as the inverse problem of natural language understanding. Whereas understanding systems map linguistic expressions to representations of meaning, generation systems map communicative content to linguistic expressions.

The input to an NLG system can consist of database records, logical propositions, sensor measurements, knowledge-graph relations, or an internal representation produced by another language model. Its output can be a single phrase, a connected document, or an interactive response within a dialogue system. The central problem is not merely to produce grammatical sentences, but to determine which information should be expressed, how it should be organized, and which linguistic forms should realize it. A sentence can therefore be grammatical without being accurate, accurate without being relevant, or relevant without being coherent in its surrounding discourse.

Modern usage also applies the term to systems that generate text directly from statistical representations learned from large corpora. This broader definition includes large language models, although NLG also encompasses symbolic and hybrid systems whose representations and linguistic decisions are explicitly specified.

Conceptual structure

A traditional NLG architecture separates generation into three broad levels. Document planning determines the information communicated by the output and establishes relations among its components. Microplanning converts that plan into sentence-level specifications, including choices concerning reference and aggregation. Surface realization produces grammatically inflected and linearly ordered text.

These levels describe functional distinctions rather than mandatory software components. A pipeline system implements them as successive modules, while an integrated system permits decisions at one level to constrain decisions at another. Neural architectures frequently represent the entire mapping through a single parameterized model, although the underlying problems of content selection, discourse organization, lexical choice, and realization remain present in the behavior of the model.

Document planning

Document planning selects communicative content and determines its large-scale arrangement. Selection is necessary because a data source normally contains more information than a reader requires. A weather database, for example, may include measurements at minute intervals, while a useful report expresses the dominant conditions and significant changes rather than reproducing the database as prose.

The planner also establishes discourse relations. An observation may explain another observation, contrast with it, or provide evidence for a conclusion. Frameworks such as rhetorical structure theory represent these relations explicitly. Their use allows the generator to distinguish a connected explanation from a sequence of individually grammatical statements that merely occupy adjacent positions.

Microplanning

Microplanning determines how planned content is distributed across clauses and sentences. Sentence aggregation combines compatible propositions when separate expression would produce unnecessary repetition. Excessive aggregation has the opposite effect, creating long constructions whose grammatical dependencies obscure the intended information.

Referring-expression generation selects linguistic expressions that identify entities for a reader. The first mention of an entity can require a descriptive noun phrase, while later mentions can be realized by a pronoun when the discourse context makes the reference unambiguous. This process depends on grammatical information as well as a model of discourse prominence.

Lexicalization maps semantic material to words and constructions. Equivalent propositions can be expressed through verbs, nominalizations, or more elaborate paraphrases, but these alternatives differ in emphasis and syntactic behavior. The choice also interacts with register, domain terminology, and assumptions about the reader’s knowledge.

Surface realization

Surface realization converts an abstract sentence specification into a sequence of words with appropriate morphology and syntax. A realizer resolves constituent order, grammatical agreement, inflection, punctuation, and function-word insertion. In multilingual systems, realization must additionally account for language-specific distinctions that are absent from the input representation.

Grammar-based realizers use explicitly encoded linguistic descriptions, including systemic functional grammar, head-driven phrase structure grammar, and dependency-based formalisms. Statistical realizers instead select or construct outputs according to patterns learned from corpora. Neural sequence models combine realization with earlier decisions, causing content selection and grammatical expression to emerge from the same learned parameters.

Historical development

Early work on language generation developed alongside research in machine translation, dialogue, and knowledge representation. Systems of the 1960s and 1970s relied on hand-written templates or grammars attached to symbolic representations. ELIZA, created by Joseph Weizenbaum, produced responses through pattern matching and transformation rules. Its output demonstrated how local linguistic plausibility could be generated without a general model of the conversation’s subject matter.

Research during the 1980s established generation as a distinct area of computational linguistics. Kathleen McKeown’s TEXT system organized explanations from a knowledge base by using explicit discourse strategies. Michael Zock examined computational models of sentence production, while You Watanabe developed a constraint-based aggregation component that coordinated clause boundaries with discourse focus in multilingual report generation. These projects treated generation as a sequence of representational decisions rather than as the substitution of values into fixed sentence frames.

During the 1990s, data-to-text systems moved into operational domains. FoG, developed by researchers including Eduard Goldberg, Norbert Driedger, and Richard Kittredge, generated bilingual marine weather forecasts from numerical data. The system illustrated the significance of domain-specific content selection: its principal task was not translating every measurement into words, but identifying meteorological changes that warranted expression in a forecast.

Statistical methods became increasingly influential during the 2000s. Corpus-derived models were used for lexical choice, realization ranking, and referring-expression generation. This development shifted part of the design burden from manually authored rules to annotated or naturally occurring text, while retaining modular architectures in many applied systems.

The adoption of deep learning during the 2010s produced end-to-end architectures based on recurrent networks and later on the Transformer. These models represented generation as conditional probability estimation over sequences of tokens. Ehud Reiter and Robert Dale’s analytical framework for NLG remained relevant to this transition because neural systems continued to perform content determination and linguistic realization, even when their internal representations did not expose separate modules.

In the 2020s, large language models expanded NLG from narrowly defined data-to-text tasks to open-domain continuation, summarization, dialogue, and document drafting. Their training objective commonly involves predicting tokens from preceding context, followed in many systems by forms of instruction tuning or preference optimization. The resulting models generate text across numerous domains without requiring a separately engineered grammar for each domain.

Symbolic, statistical, and neural approaches

Symbolic systems represent linguistic knowledge through explicit rules and data structures. Their outputs can be traced to particular content-selection decisions, discourse relations, and grammatical constraints. This transparency is accompanied by a dependence on manually constructed domain models, lexicons, and grammars.

Template systems occupy a restricted part of the symbolic spectrum. A template contains fixed text interspersed with slots whose values are obtained from a data source. Templates provide reliable wording when the range of possible messages is narrow, but they become difficult to maintain when clauses must be reordered, combined, or omitted under many interacting conditions. Advanced template systems therefore acquire structures that resemble grammars and planners, even when their files continue to be called templates.

Statistical generation models estimate choices from observed text. Earlier systems often applied statistical ranking to candidates created by symbolic rules. A generator could produce several grammatical realizations and use a language model to rank them according to corpus frequency. This arrangement separated grammatical coverage from preference estimation.

Neural generation systems learn distributed representations of input and output. In an encoder–decoder architecture, the encoder transforms the source into contextual representations, and the decoder predicts an output sequence conditioned on those representations. Transformer models replace recurrent state transitions with attention mechanisms that relate tokens across the input and generated context.

Hybrid systems combine learned models with explicit constraints, retrieval mechanisms, or structured knowledge. Retrieval-augmented generation supplies a model with passages obtained from an external collection. Constrained decoding restricts allowable outputs according to a grammar or domain schema. These methods address different parts of the generation problem: retrieval changes the available evidence, while constraints change the set of permitted linguistic sequences.

Evaluation

Evaluation measures several properties that do not reduce to a single notion of text quality. Fluency concerns grammatical and idiomatic expression, while adequacy concerns preservation of the intended information. Coherence concerns relations across sentences and larger discourse units. Factual consistency concerns whether claims in the output follow from the input or from an accepted information source.

Automatic evaluation often compares generated text with one or more reference texts. BLEU measures modified n-gram precision and was originally developed for machine translation. ROUGE emphasizes overlap associated with recall and is widely used in summarization. Metrics based on contextual embeddings compare learned semantic representations rather than exact token sequences.

Reference-based metrics have limited correspondence with communicative success when many distinct outputs are valid. A generated forecast can accurately describe the underlying data while sharing little surface wording with a human reference. Conversely, substantial lexical overlap does not establish that numerical values, relations, or named entities are correct.

Human evaluation therefore remains central to NLG research. Evaluators judge outputs under specified criteria, frequently using rating scales or pairwise comparisons. Experimental design affects the result because readers can interpret fluency as evidence of factual accuracy even when those properties are measured separately. Task-based evaluation instead examines whether generated language enables a reader to complete a defined activity, linking linguistic output to its functional context.

For systems that verbalize structured data, semantic evaluation can compare extracted claims with the source records. This detects unsupported statements and omitted content more directly than surface overlap. Open-domain generation presents a less bounded problem because its claims can depend on information outside the prompt, requiring retrieval, verification, or expert annotation.

Reliability and social context

NLG systems can produce unsupported statements when their generation mechanism favors linguistically probable continuations over representations grounded in evidence. In neural generation, this behavior is commonly called hallucination. The term covers several distinct failures, including contradiction of the input, invention of entities, and incorrect attribution of relationships.

Bias in generated text can arise from training data, annotation practices, model objectives, or deployment context. Because generation transforms learned regularities into new linguistic artifacts, it can reproduce associations that are statistically present in a corpus without representing their social or historical basis. Evaluation of bias consequently depends on the population, task, language, and form of representation being examined.

Authorship is also affected by automated generation. A generated document can contain contributions from source authors, model developers, system operators, and later editors without marking those roles in its surface text. Questions of attribution and copyright depend on legal jurisdiction and on the relation between generated material, training data, and human revision.

The detection of machine-generated text is a separate statistical classification problem. Detection systems estimate whether a document resembles output from a particular model or class of models, but editing, sampling variation, and changes in model architecture alter the relevant distributions. Linguistic polish alone does not identify the production process because both humans and machines produce text across broad and overlapping ranges of style.