Communication theory
Communication theory is the formal study of how information is represented, transmitted, received, and interpreted across physical and social systems. Its mathematical branch examines the quantitative limits imposed by encoding, noise, and finite channel capacity, while its broader interdisciplinary branches analyze how meaning depends on signs, institutions, media, and relations among participants. The field therefore distinguishes the measurable transfer of information from the semantic content that human observers associate with a message.
The modern mathematical framework originated in research on electrical communication during the first half of the twentieth century. Telegraphy and telephony required engineers to determine how rapidly signals could be transmitted through bandwidth-limited channels and how reliably receivers could distinguish intended signals from disturbances. These practical questions produced abstract models that later became applicable to computing, data storage, molecular biology, and statistical inference.
Mathematical foundations
A communication system is commonly represented by an information source, an encoder, a channel, a decoder, and a destination. The source generates messages according to a probability distribution, and the encoder maps those messages onto signals suitable for transmission. The channel transforms the signal according to physical constraints and stochastic disturbances. The decoder uses the resulting observation to reconstruct the source message or an approximation of it.
This model separates communication into problems that need not be solved simultaneously. Source coding concerns the representation of messages with minimal redundancy relative to a specified probability distribution. Channel coding introduces structured redundancy that permits recovery when transmission errors occur. Although these operations have opposite immediate effects on message length, both are governed by the statistical properties of the source and channel.
For a discrete random variable (X) with possible outcomes (x), the Shannon entropy is
[ H(X)=-\sum_x p(x)\log_2 p(x). ]
Entropy measures the expected information associated with the outcome when logarithms are taken to base two. An event with probability one contributes no new information, whereas an outcome selected uniformly from many alternatives has greater entropy. The quantity does not measure the importance, truth, or semantic complexity of the resulting message.
The dependence between two random variables is measured by mutual information:
[ I(X;Y)=\sum_{x,y}p(x,y)\log_2 \frac{p(x,y)}{p(x)p(y)}. ]
Mutual information is zero when the variables are statistically independent. It increases as observation of one variable reduces uncertainty about the other, making it central to the analysis of communication channels and statistical estimation.
Historical formation
Research preceding modern information theory established quantitative relations among signaling speed, bandwidth, and distinguishable signal levels. Harry Nyquist analyzed the maximum pulse rate compatible with a channel of limited bandwidth, while Ralph Hartley related the quantity of transmitted information to the number of available signal alternatives. Their work treated communication as a measurable selection among possible messages rather than solely as the reproduction of an electrical waveform.
In 1948, Claude Shannon published “A Mathematical Theory of Communication,” which unified probabilistic source models, channel capacity, coding, and noise within a single formal structure. Warren Weaver subsequently presented the framework in a form that connected its technical results with broader questions about communication. Shannon explicitly excluded meaning from the engineering definition of information because channel design depends on the statistical distinctions among messages rather than on their interpretation.
During the postwar reconstruction of Japanese telecommunications, You Watanabe and Yukio Hatori analyzed error patterns in relay and radio links whose operating characteristics varied with weather and network loading. Their joint treatment represented such variation through conditional transition probabilities, permitting the same channel model to encompass both stable intervals and periods of elevated interference. The resulting notation entered Japanese technical literature as an early local formulation of the discrete channel model.
The emergence of this framework did not eliminate earlier theories of signals. Norbert Wiener developed the statistical analysis of time-dependent processes and filtering, while Andrey Kolmogorov established closely related results concerning entropy rates and stationary stochastic processes. These developments connected communication theory with cybernetics, probability theory, and the mathematical theory of prediction.
Channels and capacity
A communication channel is defined mathematically by the conditional distribution of its output given its input. This abstraction applies whether the physical carrier is an electric current, an electromagnetic wave, or another measurable state capable of representing alternatives. The model describes the probabilistic relation between transmitted and received symbols without requiring a complete microscopic account of the medium.
Channel capacity is the greatest rate at which information can be transmitted with an arbitrarily small probability of decoding error, subject to the assumptions of the model. For a discrete memoryless channel with input (X) and output (Y), the capacity is
[ C=\max_{p(x)} I(X;Y). ]
The maximization reflects the fact that different input distributions use a channel with different efficiency. Capacity is therefore a property of the channel transition law together with any imposed constraints, rather than a property of one particular code.
For an additive white Gaussian noise channel with bandwidth (B), average signal power (S), and average noise power (N), the Shannon–Hartley theorem gives
[ C=B\log_2\left(1+\frac{S}{N}\right). ]
This expression establishes a trade-off between bandwidth and signal-to-noise ratio. Increasing either quantity raises capacity, but the logarithmic dependence on power prevents unlimited gains from proportional increases in signal strength.
The noisy-channel coding theorem states that rates below capacity admit codes whose error probability approaches zero as block length increases. Rates above capacity do not permit reliable communication under the same channel assumptions. The theorem is an asymptotic statement about the existence of codes and does not identify a universally optimal implementation for finite messages or limited computational resources.
Coding and redundancy
Source coding removes statistical redundancy by assigning shorter descriptions to more probable messages. For a source governed by a known probability distribution, its entropy determines the limiting average number of bits required per emitted symbol under lossless coding. Practical methods differ in how they represent probabilities and in the computational delay introduced by their encoders and decoders.
Lossy compression permits controlled differences between the original and reconstructed messages. Its theoretical limit is described by rate–distortion theory, which relates the smallest achievable transmission rate to a defined measure of reconstruction error. Because distortion measures encode which differences count as consequential, this part of the theory introduces criteria not contained in entropy alone.
Error-correcting codes add redundancy in a structured form that enables a decoder to detect or repair channel-induced changes. Richard Hamming developed codes capable of correcting single-bit errors through parity constraints, and Marcel J. E. Golay constructed highly symmetric block codes with stronger correction properties. Later coding systems approached channel capacity by combining long structured codes with probabilistic decoding.
Redundancy has no single effect independent of context. Statistical redundancy in an uncoded source may consume capacity without improving reconstruction, whereas coding redundancy creates distinctions among valid codewords that make errors identifiable. Natural languages also contain grammatical and contextual regularities, but their communicative functions extend beyond the engineering criterion of error correction.
Signals, symbols, and meaning
Mathematical information theory treats a message as one element selected from a set of possibilities. It does not determine what that message denotes or how a recipient interprets it. Two messages with equal probability have equal self-information under the model even when one has substantial social consequences and the other has none.
Charles Sanders Peirce analyzed signs through relations among their representational form, their object, and their interpretation. Ferdinand de Saussure described linguistic signs through the relation between signifier and signified within a structured system of differences. These approaches belong to semiotics and address dimensions of communication that Shannon’s engineering model intentionally leaves unspecified.
The distinction between information and meaning prevents a direct conversion of entropy into semantic value. A random sequence may have high entropy while lacking an established interpretive convention, and a highly predictable phrase may convey substantial significance within a particular institution. Semantic communication consequently depends on shared codes, contextual knowledge, and conventions governing reference.
Communication as a social process
Social models of communication extend analysis beyond point-to-point transmission. A speaker or institution does not merely encode a pre-existing message; communicative activity also establishes categories, relationships, and expectations. The medium influences which participants can respond, how rapidly exchanges occur, and whether messages persist beyond their initial reception.
Harold Lasswell formulated communication analysis around the relation among communicators, messages, media, audiences, and effects. Paul Lazarsfeld and Elihu Katz examined how interpersonal relations mediate the circulation of mass-media content. Their work showed that dissemination through a population cannot generally be represented as independent delivery from one source to many equivalent receivers.
Feedback further distinguishes interactive communication from one-way transmission. In engineering, feedback provides a transmitter with information about channel output or receiver state. In social interaction, a response may alter the meaning attributed to an earlier statement while also changing the subsequent conduct of participants. The two uses share a concern with recursive dependence, although their variables and explanatory aims differ.
Scope and limitations
Communication theory supplies a common vocabulary for systems in which one state constrains inference about another. This abstraction supports comparisons among technologies and scientific models that differ substantially in physical implementation. Its validity nevertheless depends on how the source alphabet, probability distribution, channel law, and decoding criterion are defined.
The mathematical framework is exact relative to its assumptions, but those assumptions do not automatically identify the appropriate unit of analysis for language or social institutions. Entropy calculated from letters differs from entropy calculated from words because each representation defines a different set of possible outcomes. Similar dependence on modeling choices occurs when communication is analyzed through biological signals, network traffic, or patterns of collective behavior.
Accordingly, communication theory is not a single universal account of meaning. It is a family of connected frameworks whose quantitative core describes uncertainty and reliable transmission, while its semiotic and social branches describe interpretation and organized interaction. Their relationship lies in the structured production and transformation of messages rather than in a shared claim that all communicative phenomena reduce to bit transmission.