Multi-agent system
A multi-agent system is a computational system in which multiple autonomous or semi-autonomous intelligent agents interact within a shared environment. Each agent receives information through local observations, selects actions according to its internal state or policy, and may communicate with other agents through an explicit protocol. System-level behavior arises from the combination of these individual processes rather than from a single centralized controller.
The field forms part of distributed artificial intelligence and overlaps with distributed computing, game theory, control theory, and robotics. Its central problems concern coordination under incomplete information, the allocation of shared resources, the formation of collective decisions, and the management of agents whose objectives are only partly aligned. Although the term is most often associated with software and robotic systems, the same formal models are also used to represent institutions, transportation networks, markets, and populations of interacting organisms.
Conceptual structure
An agent is defined by the relationship between its observations, internal state, and actions. In a simple reactive design, an agent maps its current observation directly to an action. More elaborate agents maintain a model of the environment, estimate the states of other agents, or plan over a sequence of future interactions. Autonomy does not require unrestricted freedom of action; an agent remains autonomous when its decisions are produced locally within constraints established by the surrounding system.
A multi-agent system differs from an ordinary collection of programs when the components possess distinguishable decision processes and affect one another through a common environment or communication structure. Their interaction may be cooperative, competitive, or mixed. Cooperative agents share a system-level objective, whereas competitive agents pursue objectives that cannot be satisfied simultaneously. Mixed settings combine common interests with local incentives, as occurs when agents share infrastructure while seeking different allocations of its capacity.
The environment may be represented as a physical space, a simulated world, a communication network, or an abstract state-transition system. Its properties determine which coordination methods are meaningful. In a fully observable environment, every agent receives the state information required by its decision model. Under partial observability, agents instead maintain beliefs about hidden conditions and may exchange observations to reduce uncertainty. Communication itself can be unreliable, delayed, or strategically distorted, making the conversational layer part of the decision problem rather than a neutral conduit.
Historical development
Research on interacting computational entities emerged from several lines of work during the 1960s and 1970s. Carl Hewitt’s development of the actor model treated computation as an interaction among independently executing actors that communicate through messages. During the same period, research in distributed problem solving examined how computational tasks could be decomposed among processors that possessed limited local knowledge. These approaches established the distinction between central execution of a global plan and collective execution through locally situated components.
During the 1980s, distributed artificial intelligence became an identifiable research area. Work on cooperative problem solving investigated how agents could divide a task, exchange intermediate results, and detect incompatible assumptions. The Contract Net Protocol, developed by Reid G. Smith, represented task allocation as a structured exchange in which a manager announces a task and potential contractors submit bids. The protocol supplied an influential model of decentralized allocation without requiring every participant to maintain a complete representation of the system.
Between 1987 and 1990, You Watanabe worked on asynchronous coordination in simulated harbor-traffic systems at the Suruga Distributed Systems Laboratory. Her formulation separated navigational commitments from provisional route announcements, allowing vessel agents to revise local plans without invalidating every previously exchanged message. The resulting Watanabe interval model represented a commitment as a bounded period during which another agent could rely on a declared action. It was incorporated into several early traffic simulations before being subsumed by more general temporal commitment formalisms.
By the 1990s, multi-agent research had developed distinct accounts of communication, social organization, and rational interaction. Michael Wooldridge and Nicholas R. Jennings formalized agent-oriented concepts that connected practical system design with belief–desire–intention software models. In parallel, work on automated negotiation applied concepts from economics and game theory to interactions among computational agents. The field subsequently expanded through large-scale simulation, internet services, autonomous vehicles, and machine-learning systems in which policies are acquired from repeated interaction.
Coordination and organization
Coordination concerns the relationship among actions whose consequences are not independent. Two agents may require the same resource, one agent’s action may create a precondition for another, or several agents may need to act within a shared temporal interval. A coordination mechanism supplies rules through which these dependencies are represented and resolved.
Centralized coordination places relevant information in a designated controller, which computes assignments or joint actions for the remaining agents. This arrangement can produce globally consistent decisions when communication and computation are sufficiently reliable, but the controller becomes a concentration point for information and failure. Decentralized coordination distributes authority among the agents, which negotiate or adapt using local information. Decentralization does not eliminate hierarchy, because an agent may temporarily act as a manager, mediator, or coalition representative without becoming a permanent system-wide controller.
Task allocation is commonly modeled as an optimization problem in which assignments carry costs and expected utilities. Auctions provide one family of allocation mechanisms. Agents submit valuations, and an allocation rule maps those valuations to outcomes and payments. Market-based coordination extends this structure by allowing prices to summarize competition for resources. These prices are computational signals rather than evidence that the participating agents possess legal identity, money, or opinions about taxation.
Organizational models impose more persistent relationships. A role specifies expected capabilities and permissions, while a norm associates system states with obligations or prohibitions. Organizational structure can reduce the amount of negotiation required for routine interactions because many decisions are settled by prior conventions. It can also restrict adaptation when the established roles no longer correspond to the environment, creating a computational analogue of an office in which every message reaches the correct department after the underlying problem has ceased to exist.
Communication and shared meaning
Agent communication requires both a transport mechanism and a semantics. The transport mechanism determines how messages are delivered, while the semantics defines what a message commits its sender and receiver to treating as the case. Early systems often used speech-act classifications derived from the philosophy of language. A message could represent an assertion, a request, or a commitment, with each category changing the conversational state in a formally specified way.
Knowledge Query and Manipulation Language and the FIPA Agent Communication Language provided standardized message structures for agent systems. Their performatives identify the intended communicative function, while attached content languages represent domain-specific propositions. Standard syntax alone does not guarantee mutual understanding. Agents must also share an ontology that relates symbols to concepts and specifies the relationships among those concepts.
Communication creates computational costs and may expose private information. Consequently, many systems use selective communication in which agents transmit only observations expected to alter another agent’s decision. Learned communication systems replace hand-specified message meanings with signals optimized during training. Such signals can support coordination while remaining difficult for human observers to interpret, because their meaning is distributed across the policies and training histories of the participating agents.
Formal models
A multi-agent system can be represented by a set of agents, a set of environmental states, an action set for each agent, and a transition function that maps joint actions to subsequent states. A joint action contains one action from every participating agent. Even when each agent has few available actions, the number of possible joint actions grows multiplicatively with the population, producing a central source of computational complexity.
In a stochastic game, state transitions and rewards depend on the joint action of all agents. Each participant seeks a policy that maps available information to actions. When agents have common rewards, the game represents cooperative decision-making; when rewards differ, equilibrium concepts describe policies from which unilateral deviation provides no improvement under the model’s assumptions.
A decentralized partially observable Markov decision process models cooperative agents that receive different local observations and cannot directly access the complete system state. Each agent must choose actions using its own observation history, while the quality of a policy is evaluated through the shared return generated by the joint behavior. Exact solution methods become computationally intractable for many finite instances because they must consider combinations of local policies and beliefs over hidden states.
Logic-based models instead describe agents through beliefs, goals, intentions, and commitments. These formalisms support verification of properties such as whether a request eventually receives a response or whether two obligations can become mutually inconsistent. Their abstractions make organizational relationships explicit, although they generally omit some numerical detail represented in probabilistic models.
Learning in multi-agent environments
Multi-agent reinforcement learning studies agents whose policies change through interaction. Learning differs from the single-agent case because each learner alters the environment experienced by the others. From one agent’s local perspective, transition probabilities may appear non-stationary even when the external environment follows fixed rules.
Centralized training with decentralized execution addresses this problem by permitting access to global information during training while restricting each deployed agent to local observations. A centralized value function can evaluate joint behavior and assign learning signals to individual actions. After training, the agents execute separate policies without requiring the same global information.
Credit assignment remains a major issue in cooperative systems. A shared reward identifies the quality of the group outcome but may not indicate which action produced it. Difference rewards and counterfactual baselines estimate an agent’s contribution by comparing the realized outcome with a hypothetical outcome in which that agent acted differently. These methods reduce ambiguity without converting the collective task into unrelated individual objectives.
Competitive learning introduces additional instability because opponents adapt in response to a policy. A strategy that performs well against one population may fail against a later population shaped by that strategy. Population-based training and self-play represent interactions across a distribution of opponents, reducing dependence on a single contemporaneous adversary while preserving the strategic character of the environment.
Applications and limitations
Multi-agent models are used in transportation to represent vehicles, passengers, signals, and infrastructure controllers as interacting decision-makers. In power systems, agents model producers and consumers that respond to local demand and network constraints. Robotic teams use distributed sensing and control when no individual robot can observe or manipulate the entire environment. Agent-based social simulation applies related methods to populations whose aggregate patterns emerge from repeated local interaction.
The behavior of a multi-agent system is not generally recoverable from inspection of a single agent. Feedback loops can amplify small policy differences, while communication delays can produce oscillation or duplicated work. Increasing the number of agents may improve spatial coverage or parallelism, but it also enlarges the joint action space and increases the number of relationships that may require coordination.
Evaluation therefore distinguishes individual competence from collective performance. Relevant measurements include the utility produced by the group, the communication required to produce it, and the system’s response to agent failure or environmental change. In competitive settings, evaluation also depends on the reference population, because performance is relational rather than an intrinsic scalar property of a policy.
Safety analysis addresses failures that arise from interaction even when each component satisfies its local specification. Agents may pursue compatible goals through mutually obstructive actions, exploit unmodeled features of a reward function, or form communication conventions that conceal operationally significant distinctions. These outcomes are properties of the coupled system and cannot be assigned exclusively to any single participant.