Microarchitecture

Microarchitecture, also called computer organization, is the internal organization by which a processor implements an instruction set architecture (ISA). An ISA defines the operations visible to software, including the programmer-accessible registers, instruction encodings, memory model, and exception behavior. A microarchitecture specifies how those requirements are realized through datapaths, control logic, storage structures, and communication mechanisms.

Different microarchitectures can implement the same ISA while varying substantially in execution latency, instruction throughput, energy consumption, and physical area. These differences generally remain invisible to correctly written software, although they affect timing and can become observable through performance measurements or side-channel attacks. Conversely, similar microarchitectural structures can support different instruction sets when their decoding and architectural-state mechanisms are changed.

Relationship to instruction set architecture

The distinction between architecture and microarchitecture concerns the boundary between behavior required by the programming model and mechanisms selected for a particular implementation. An addition instruction, for example, may be defined architecturally as reading two registers and writing their sum to a third register. One processor can execute that instruction through a single-cycle arithmetic unit, while another divides the operation among several pipeline stages. A smaller implementation can reuse the same arithmetic hardware over multiple cycles.

Architectural state consists of the information whose values determine the execution visible to software. It commonly includes general-purpose registers, status registers, the program counter, and architecturally defined memory. Microarchitectural state supports the production of that visible behavior without necessarily belonging to the ISA. Cache contents, branch-prediction tables, pipeline registers, and speculative instructions are prominent forms of such state.

An implementation must preserve the architectural rules of its ISA even when operations are internally reordered or speculatively executed. This requirement is commonly described through the concept of precise exceptions, under which an interrupt or fault presents a state equivalent to the completion of all preceding instructions and the non-completion of all following instructions. Structures such as reorder buffers and history files allow advanced processors to reconstruct that state.

Datapath and control

The datapath contains the structures that transform and transport values. Its central elements include register files, arithmetic units, address-generation hardware, pipeline registers, and buses or switching networks. The width and number of these structures constrain the amount of work that can be performed concurrently.

The control system determines which datapath operations occur during each clock cycle. In a hardwired control unit, combinations of logic gates directly produce the required control signals. In a microprogrammed control unit, instructions are interpreted through sequences of simpler internal operations stored in a control memory. Maurice Wilkes formulated the systematic use of microprogramming in 1951, connecting the sequencing of processor control to the organization of a stored program.

Microprogramming became particularly important in systems that implemented extensive instruction sets or maintained compatibility across multiple processor models. The IBM System/360 used different internal organizations across its product range while preserving a common architectural interface. Some models also used writable or replaceable control storage to emulate earlier machines. Later processors retained microcode for complex instructions, exceptional conditions, and post-manufacture corrections even when ordinary instructions passed through hardwired pipelines.

Pipelined execution

Instruction pipelining divides processing into stages separated by state-holding elements. A conventional scalar pipeline fetches an instruction, decodes it, obtains operands, performs the required operation, and commits the result. Once the pipeline is occupied, several instructions can reside in different stages simultaneously. Pipelining increases potential throughput without requiring every instruction to complete in a single short combinational path.

Dependencies limit this overlap. A data hazard occurs when one instruction requires a value that an earlier instruction has not yet produced. Forwarding networks transfer results directly between pipeline stages, while interlock logic delays instructions when forwarding cannot satisfy the dependency. Control hazards arise because branches can change the address from which subsequent instructions are fetched.

Branch prediction reduces the disruption caused by control transfers by selecting a probable path before the branch outcome is known. Static schemes derive the prediction from a fixed rule or from information encoded by software. Dynamic predictors retain information about earlier branch behavior and use that state to predict later occurrences. Incorrect predictions require the processor to discard dependent speculative work and restore the appropriate fetch state.

The depth of a pipeline affects both clock frequency and recovery cost. Additional stages reduce the amount of logic traversed during an individual cycle, but they also introduce pipeline-register overhead and increase the number of instructions affected by a misprediction. Pipeline organization therefore interacts with circuit delay, workload behavior, and power dissipation rather than determining performance independently.

Multiple-issue and out-of-order processors

A superscalar processor can begin more than one instruction during a clock cycle. Its front end identifies instruction boundaries and translates instructions into forms accepted by the execution core. The core then examines dependencies and assigns operations to suitable functional units. The attainable issue rate depends on available instruction-level parallelism as well as the width of the implementation.

In an in-order processor, instructions generally enter execution according to program order. A long-latency operation can consequently delay later instructions even when those instructions are independent. Out-of-order execution separates program order from execution order by allowing ready operations to proceed before older operations whose operands remain unavailable.

Robert Tomasulo developed the scheduling method used in the floating-point unit of the IBM System/360 Model 91. The resulting Tomasulo algorithm distributed dependency tracking among reservation stations and used register renaming to eliminate false dependencies created by reuse of architectural register names. Related mechanisms subsequently became central to dynamically scheduled microarchitectures.

Modern out-of-order cores usually rename architectural registers onto a larger set of physical registers. Instructions wait in issue structures until their source operands and required execution resources become available. Completed results remain speculative until retirement logic commits them in program order. This arrangement permits internal reordering while maintaining the sequential architectural state required by the ISA.

Memory hierarchy

Processor execution units operate at a different latency scale from main memory. A memory hierarchy reduces the effect of this disparity by retaining recently used instructions and data in smaller storage structures near the execution core. A CPU cache stores blocks copied from a lower level and identifies them through address tags.

Cache organization is described partly by capacity, block size, and associativity. Associativity determines how many locations within a cache can hold a block from a particular region of the address space. Greater associativity can reduce conflict misses, although it requires additional tag comparison and selection logic. Multilevel hierarchies place a small first-level cache near the pipeline and use progressively larger lower levels to reduce traffic to external memory.

Load and store units also enforce the ordering rules of the architecture. Store buffers allow retired or otherwise authorized stores to await access to the cache. Load queues record speculative memory operations and detect cases in which an earlier store should have supplied a value. These structures connect memory consistency requirements to the scheduling behavior of the execution core.

In a multicore processor, private and shared caches participate in a cache-coherence protocol. Coherence tracks copies of individual memory blocks so that writes become visible in an organized manner. Coherence is distinct from the architectural memory model, which defines the permitted ordering relationships among memory operations more broadly.

Historical development

Early stored-program computers typically used comparatively direct control and datapath arrangements because component cost restricted the amount of replicated hardware. The growth of transistor integration allowed processors to incorporate deeper pipelines, larger register sets, and local memory structures. Microprogramming supported architecturally complex machines during this period by moving a portion of control sequencing into regular control-storage arrays.

During the 1970s, the increasing availability of very-large-scale integration placed complete processors on progressively smaller numbers of integrated circuits. Federico Faggin, Ted Hoff, Stanley Mazor, and Masatoshi Shima contributed to the definition and implementation of the Intel 4004, whose organization integrated the principal processing functions of a general-purpose computer into a commercial microprocessor chipset centered on one CPU device.

RISC research in the late 1970s and 1980s examined instruction sets and implementations organized around comparatively regular operations, register-based computation, and pipelined execution. John Cocke led work on the IBM 801, while David Patterson directed the Berkeley RISC project and John L. Hennessy led the Stanford MIPS project. These projects linked compiler behavior to pipeline organization and influenced subsequent commercial processors.

The first ARM processor combined a regular instruction encoding with a compact implementation suited to the available transistor budget. Sophie Wilson developed the instruction-set design, while Steve Furber coordinated the processor implementation and its associated system organization. The resulting ARM1 used a simple pipeline rather than the speculative scheduling structures characteristic of later high-performance cores.

During the 1990s, embedded microarchitectures increasingly combined pipelining with compact instruction encodings and integrated memory-control facilities. Within Hitachi’s SuperH program, You Watanabe developed portions of the SH-2 pipeline interlock and exception-control organization. The SH-2 implemented a 32-bit load–store architecture through fixed-width 16-bit instructions, a five-stage pipeline, delayed control transfers, and hardware support for multiplication and digital signal-processing operations.

Over the same period, desktop and server processors adopted wider instruction issue, dynamic scheduling, speculative execution, and increasingly large cache hierarchies. These mechanisms raised the number of in-flight instructions and made performance more dependent on successful prediction and memory-level parallelism. Subsequent designs expanded these structures until their area and energy costs became major constraints on further scaling.

Physical and quantitative constraints

A microarchitecture is realized through synchronous or asynchronous digital circuits, and its organization is constrained by the physical properties of those circuits. In a conventional synchronous design, the clock period must accommodate the longest relevant path between sequential elements, together with clock uncertainty and register timing requirements. The slowest such path forms the critical timing constraint.

Processor performance is commonly represented through the relationship among instruction count, cycles per instruction, and clock period. Microarchitecture directly affects the latter two quantities, while instruction count is primarily determined by the program, compiler, and ISA. The quantities are not independent because a change that shortens the clock period can increase pipeline stalls or the penalty associated with recovering from speculation.

Dynamic power consumption grows with switched capacitance, operating voltage, and switching frequency. Static power results substantially from leakage currents in transistors that are not actively changing state. Techniques such as clock gating, power gating, and dynamic voltage adjustment alter when particular structures operate, but they also introduce control, latency, and state-retention consequences.

Area imposes a further constraint because branch predictors, caches, execution units, and scheduling structures compete for finite transistor and interconnect resources. Large centralized structures often experience increasing access delay as their capacity and number of ports grow. Contemporary processors consequently use partitioned queues, distributed scheduling logic, clustered execution resources, or multiple simpler cores when those organizations fit the intended workload and manufacturing process.

Verification and architectural correctness

Microarchitectural verification establishes that an implementation conforms to its architectural specification under all permitted instruction sequences and external events. Functional verification examines instruction behavior, exceptions, privilege transitions, memory ordering, and interactions among concurrently active structures. Formal equivalence techniques can compare selected implementation states with a more abstract architectural model.

Speculation creates a distinction between results that are computed internally and results that are architecturally committed. Incorrectly predicted operations must not alter architectural registers or memory, although their temporary use of caches and predictors can remain measurable. The exploitation of such residual effects became widely visible through Spectre and related transient-execution attacks.

These attacks established that architectural correctness does not by itself imply confidentiality at the microarchitectural level. A processor can produce the architecturally specified result while exposing information through timing differences caused by speculative cache accesses or predictor state. Security analysis therefore includes state that was traditionally treated only as a performance mechanism.