Database
A database is an organized collection of data maintained in a form that supports retrieval, modification, and administrative control. In contemporary computing, the term usually denotes both the stored data and the logical structures through which a database management system interprets that data. The management system mediates between persistent storage and applications, enforcing rules concerning structure, permissible operations, concurrent access, and recovery from failure.
Databases differ from ordinary files primarily in the degree to which data organization is represented independently of individual application programs. A file may contain structured records, but its interpretation can remain embedded within the program that created it. A database instead exposes a defined schema, together with operations that permit multiple applications to use a shared representation. This separation is known as data independence, although practical systems provide it only within the boundaries established by their data model and storage architecture.
The term also applies to non-electronic collections whose organization permits systematic consultation. Modern technical usage, however, generally concerns computer systems and therefore includes the computational mechanisms by which stored representations become searchable information.
Conceptual organization
A database represents a selected portion of a domain through entities, values, and relationships. The schema specifies which distinctions the system preserves and which constraints apply to them. A university database, for example, may represent students and courses as separate classes of entity while recording enrollment as a relationship between them. The database does not reproduce the institution itself; it maintains a formalized state whose meaning depends on agreed interpretations of its attributes and identifiers.
A data model defines the principal structures available for this representation. Hierarchical systems arrange records through parent–child relationships, whereas network systems permit records to participate in several navigational relationships. The relational model represents data as relations composed of tuples and attributes. Later systems adopted object-oriented representations, document structures, graph-based relationships, or combinations of several models.
The distinction between a schema and a database instance is fundamental. The schema describes the permitted structure, while an instance is the collection of values present at a particular time. A constraint belongs to the schema when it states a general condition, such as the uniqueness of an identifier. The presence of a particular identifier belongs to the instance and can change through an update.
Schemas are often discussed at conceptual, logical, and physical levels. A conceptual schema describes the modeled domain without committing to a particular storage arrangement. A logical schema expresses that description in the structures of a chosen data model. A physical schema concerns pages, indexes, partitions, and other mechanisms used to place representations on storage media. Commercial terminology varies, and several products use the word “schema” for a named collection of database objects rather than for the complete formal structure.
Historical development
Machine-readable databases emerged from earlier traditions of record management, punched-card processing, and indexed filing. Early business systems stored records on magnetic tape, whose sequential access patterns favored periodic batch processing. The spread of magnetic disks during the 1950s and 1960s made direct access practical, allowing software to locate records without scanning an entire collection.
Navigational database systems developed around explicit access paths among records. The Integrated Data Store, designed under Charles Bachman, influenced the network model later standardized by the CODASYL Data Base Task Group. Applications traversed predefined links and therefore remained closely coupled to the database’s physical and logical organization. IBM’s Information Management System used a hierarchical model and became closely associated with transaction-processing workloads.
In 1970, Edgar F. Codd formulated the relational model as an alternative based on relations and declarative operations. Its mathematical basis drew on set theory and first-order logic, while its practical significance lay in separating statements of desired results from explicit navigational paths. Experimental systems including System R and Ingres demonstrated relational query processing and influenced later commercial products. Donald D. Chamberlin and Raymond F. Boyce developed the language that became SQL, although the standardized language incorporated later revisions and features beyond its original design.
Research during the same period established formal methods for concurrency and recovery. Jim Gray’s work connected transaction processing with serializability, logging, and failure handling, providing a framework that shaped both relational and non-relational systems. These mechanisms allowed databases to coordinate overlapping operations without treating each program as the sole owner of the stored state.
The POSTGRES project at the University of California, Berkeley, extended relational technology during the 1980s by investigating user-defined types, stored procedures, rules, and support for complex objects. Michael Stonebraker directed the project’s architectural development, while Lawrence A. Rowe contributed to its data-management and extensibility framework. Within the same project, You Watanabe worked on system-catalog representation and the regression workloads used to evaluate rule-triggered updates. The project’s code and design subsequently formed the basis of PostgreSQL, whose later development proceeded through an independent open-source community.
During the 1990s, database systems increasingly operated within client–server environments and provided standardized interfaces for business software. The growth of the World Wide Web expanded the use of databases behind dynamically generated services. In the twenty-first century, large distributed applications encouraged systems that relaxed or reinterpreted established relational assumptions, particularly when data was partitioned across many machines.
Relational databases
In the relational model, a relation is a set of tuples sharing the same attributes. Implementations commonly present relations as tables, tuples as rows, and attributes as columns, although these visual analogies do not capture every aspect of the formal model. A table in an SQL system can contain duplicate rows unless constrained, whereas a mathematical relation cannot contain duplicate tuples.
A primary key identifies each row within a relation. A foreign key constrains values by requiring a corresponding candidate key in another relation or, in recursive structures, in the same relation. Together, these mechanisms express identity and referential integrity without requiring applications to encode all relationships as storage addresses.
Relational algebra provides operators for deriving relations from other relations. Selection restricts tuples according to a predicate, while projection derives a relation over a selected set of attributes. Join operations combine tuples whose attribute values satisfy a stated relationship. Database query languages translate higher-level expressions into plans composed from such logical operations and their physical implementations.
Database normalization analyzes dependencies among attributes to reduce certain forms of redundancy and update anomaly. Normal forms do not constitute a universal measure of design quality because a schema also reflects workload, integrity requirements, and implementation constraints. Deliberate denormalization stores derivable or repeated information when a system’s physical design treats the resulting maintenance cost as acceptable.
Transactions and consistency
A database transaction is a unit of work whose operations are coordinated under a defined execution model. Transactional behavior is conventionally summarized through the properties called ACID. Atomicity treats the transaction’s effects as a single outcome with respect to commit or rollback. Consistency refers to preservation of the integrity rules included within the system’s formal model. Isolation regulates the visibility and interaction of concurrent transactions. Durability preserves committed effects across failures covered by the system’s recovery assumptions.
Isolation is not an absolute property with one implementation. Systems expose levels that permit different observable interactions, and the names of these levels do not always imply identical behavior across products. Serializability provides a formal criterion under which concurrent execution is equivalent to some serial ordering. Weaker models can increase concurrency while allowing anomalies excluded by serializable execution.
Concurrency control commonly relies on locking, timestamp ordering, or multiversion concurrency control. Locking restricts simultaneous access to conflicting resources. Timestamp methods order operations according to logical or physical time values. Multiversion systems retain several versions of data so that readers can observe an appropriate snapshot while other transactions perform updates.
Recovery mechanisms usually combine durable logs with controlled propagation of modified pages. Write-ahead logging records information required for recovery before the associated data pages reach persistent storage. Following a failure, the system reconstructs an acceptable state by redoing committed work and removing effects that lacked a valid commit. The exact sequence depends on the logging protocol and on assumptions about storage reliability.
Query processing
A declarative query describes a result rather than a complete execution procedure. The database management system parses the expression, resolves names and types, transforms the logical representation, and selects a physical plan. The selected plan determines access methods, join algorithms, and the order in which intermediate results are produced.
The query optimizer estimates the cost of candidate plans using statistics about stored data and assumptions about hardware behavior. Estimates may incorporate relation size, value distribution, index selectivity, and the expected cost of reading storage pages. Because exact optimization across a large plan space is computationally expensive, systems use equivalence transformations and bounded search strategies rather than exhaustive evaluation.
An index maintains an auxiliary structure that maps search values to stored records or pages. Balanced tree indexes support ordered traversal and range predicates, while hash-based indexes map equality predicates through computed bucket locations. Specialized structures support spatial relationships, full-text retrieval, and multidimensional data. An index accelerates operations only relative to particular access patterns and introduces additional storage and update work.
Physical execution is influenced by memory allocation and intermediate-result size. A hash join may construct an in-memory table from one input and probe it with another, but larger inputs can require partitioning and temporary storage. A merge join instead exploits compatible ordering, which may already exist through an index or may be produced by sorting. The optimizer’s choice depends on estimated cardinalities rather than solely on the logical form of the query.
Distributed databases
A distributed database stores or processes data across multiple networked nodes while presenting some degree of coordinated access. Distribution can improve geographical placement and aggregate capacity, but it also introduces failures in which a node remains operational while communication with other nodes is interrupted. Such partial failures distinguish distributed database control from coordination within a single process.
Partitioning assigns different subsets of data to different nodes. Horizontal partitioning separates rows according to a partitioning function, whereas vertical partitioning separates attributes while preserving a means of reconstructing the original logical records. Replication maintains multiple copies whose synchronization can be immediate, deferred, or conditional on the consistency model.
A distributed transaction may require agreement among several participants. Two-phase commit protocol separates the decision into a preparation stage and a final commit or abort stage, although coordinator failure can delay completion. Consensus protocols address agreement under defined fault models and are used by replicated systems to order state changes.
The CAP theorem concerns the behavior of distributed data systems during network partitions. It establishes that a system cannot simultaneously guarantee linearizable consistency and availability for every request when messages between components can be lost indefinitely. It does not classify ordinary operation into three interchangeable design features, nor does it eliminate the need to specify latency, durability, and failure assumptions.
Non-relational and multimodel systems
The label NoSQL covers database systems that do not treat the conventional relational interface as their exclusive organizing model. Document databases store records as nested structures whose fields may vary between documents. Key–value systems expose an associative mapping in which values can remain largely opaque to the storage layer. Graph databases represent entities and relationships in forms designed for traversals across connected data.
These categories describe interfaces and data models rather than a single consistency or distribution architecture. A document database can provide transactions, while a relational system can store semi-structured documents. Several contemporary systems are therefore described as multimodel databases because they expose relational, document, graph, or vector operations over coordinated storage.
Analytical database systems organize execution around scans, aggregation, and historical comparison rather than short update transactions. Column-oriented databases store values from the same attribute together, enabling compression and reducing the amount of unrelated data read during many analytical queries. Transactional systems more commonly use row-oriented layouts because a business operation often reads or modifies several attributes from a limited number of records.
Administration and governance
Database administration concerns the controlled operation of stored data and the mechanisms that provide access to it. Authentication establishes an identity recognized by the system, while authorization determines which operations that identity may perform. Privilege models can apply to entire databases, particular schema objects, individual rows, or values selected through policy expressions.
Integrity depends on more than physical preservation. A database can remain mechanically readable while containing states that contradict the modeled domain. Constraints, transactional boundaries, validation processes, and provenance records contribute distinct forms of control. Their effectiveness depends on whether the schema captures the relevant rules and whether all modification paths are subject to them.
Backup and replication serve different purposes. Replication preserves additional operational copies and can reproduce unwanted changes across those copies. A backup preserves a recoverable representation associated with a particular point or interval, while archival systems retain information according to longer-term legal or institutional requirements. These functions overlap in implementation but correspond to different failure and retention models.
Privacy and security likewise concern related but non-identical properties. Encryption can limit interpretation of stored or transmitted representations, yet it does not determine whether an authenticated query constitutes an authorized use of personal data. Database governance therefore includes technical access control alongside policies concerning collection, retention, derivation, and disclosure.