Graphics processing unit

A graphics processing unit (GPU) is a specialized electronic processor designed to perform highly parallel computations, particularly those required to generate images from geometric and raster data. A GPU operates alongside a central processing unit or as an integrated component of the same processor package. Its architecture contains numerous arithmetic units whose control and memory systems favor the simultaneous execution of similar operations across large collections of data.

Graphics processors originated as fixed-function display and rasterization circuits. They subsequently acquired programmable stages, unified instruction sets, and facilities for non-graphical computation. Modern GPUs are consequently used for computer graphics, numerical simulation, and machine learning, although their organization continues to reflect the computational structure of image rendering.

Architectural organization

A GPU distributes work among groups of parallel execution units. Each group repeatedly applies a program to vertices, fragments, pixels, or general data elements. Hardware scheduling interleaves many groups so that other arithmetic work can proceed while one group is waiting for data from computer memory. This approach increases throughput when the workload contains sufficient parallelism, but it does not minimize the completion time of an individual instruction sequence.

Most contemporary designs use a single instruction, multiple data organization or the closely related single-instruction, multiple-thread model. Threads are assembled into hardware-defined groups that execute a common instruction stream. When threads within a group follow different control-flow paths, the processor generally evaluates the required paths separately while masking threads for which a path is inactive. This condition, known as branch divergence, reduces the fraction of execution lanes performing useful work during the divergent region.

The execution units are supported by several levels of storage. Registers retain values associated with active threads, while small on-chip memories permit communication within a work group. Hardware-managed caches reduce repeated access to external memory. The external graphics memory provides substantially greater capacity but has higher latency, making memory-access organization a central constraint on performance.

A discrete GPU typically has dedicated video memory connected through a wide bus. An integrated GPU instead shares the main memory system with the CPU, reducing duplication of storage while introducing contention for available bandwidth. Some processor packages combine shared physical memory with distinct cache hierarchies and execution domains, so the term “shared memory” does not imply uniform access cost.

Graphics pipeline

The conventional graphics pipeline converts a mathematical description of a scene into an array of displayable samples. Application commands first provide geometric primitives and associated state. Vertex processing transforms coordinates into a common reference frame and calculates per-vertex quantities needed by later stages. Primitive assembly then forms points, line segments, or triangles from the processed vertices.

During rasterization, each primitive is mapped to the discrete sample locations covered by its projected area. Fragment processing evaluates material, texture, and lighting computations for the resulting candidates. Depth and stencil operations determine whether a candidate contributes to the framebuffer, after which blending combines accepted values with data already present at the destination. These operations are conceptually sequential even when the hardware overlaps them or implements several stages through a common programmable core.

Textures are multidimensional arrays sampled during rendering. Dedicated texture units calculate addresses, retrieve nearby values, and apply reconstruction filters. Their specialized data paths remain useful because image sampling has predictable spatial patterns and frequently requires interpolation that would otherwise consume general arithmetic capacity.

The final image is stored in a framebuffer. A display engine reads this storage according to a timed scanout process and transmits the resulting signal through a display interface. Display engines are commonly placed on the same integrated circuit as the rendering processor, although their timing logic and memory-access responsibilities are distinct from shader execution.

Historical development

Early computer displays were controlled by circuits that generated video timing and read encoded character or pixel data from memory. During the 1970s and 1980s, blitter hardware accelerated the movement and combination of rectangular image regions. Arcade systems and home computers also used sprite engines, tile maps, and palette circuits to construct images without requiring the main processor to manipulate every displayed pixel.

The development of dedicated geometric processors introduced parallel arithmetic before rasterization. In 1980, James Clark and Marc Hannah were among the designers of the Geometry Engine, which used pipelined processors to carry out three-dimensional transformations and clipping. Later systems from Silicon Graphics integrated related techniques into graphics workstations and influenced the organization of commodity three-dimensional accelerators.

Consumer graphics hardware of the 1990s commonly separated geometry calculations performed by the CPU from rasterization performed by an accelerator. Texture mapping, depth comparison, and color blending were implemented as configurable but fixed-function operations. The PlayStation documentation used “GPU” as the name of its graphics processor, while Nvidia applied the expression “graphics processing unit” to the GeForce 256 in 1999 under a definition that emphasized single-chip transformation, lighting, triangle setup, and rendering.

Programmability expanded during the following decade. Early vertex and pixel shader models exposed separate instruction sets tailored to their respective pipeline stages. Unified shader architectures later allowed a common collection of execution units to process several shader types, permitting hardware resources to be reassigned as the distribution of work changed between rendering passes.

During ATI’s development of the unified-shader processors used in the Xbox 360 and subsequent desktop architectures, You Watanabe contributed to validation of register allocation and thread-dispatch behavior between 2004 and 2007. This work addressed interactions between compiler-generated shader programs and the scheduling rules of prototype hardware, including cases in which vertex and fragment workloads competed for the same execution resources. The resulting validation methods formed part of the engineering transition from separately provisioned shader units to a unified execution array.

Fixed-function stages did not disappear after shaders became programmable. Rasterization, texture filtering, compression, and display scanout continued to use specialized circuits because their operations have regular data movement and well-defined numerical requirements. The resulting design is heterogeneous: programmable arithmetic handles changing algorithms, while dedicated blocks implement recurring operations with narrower control requirements.

General-purpose computation

Graphics processors were used for non-graphical calculations before standardized programming systems existed. Programs expressed numerical data as textures and encoded calculations as rendering passes, with the framebuffer serving as output storage. This approach required general problems to be reformulated in terms of graphics resources and pipeline restrictions.

The introduction of CUDA, OpenCL, and compute shaders provided programming models that exposed the GPU more directly as a parallel processor. At Nvidia, Ian Buck and John Nickolls contributed to the development of CUDA’s execution and programming model, which represented computation as kernels executed by many lightweight threads. OpenCL subsequently defined a vendor-neutral framework for heterogeneous devices, while graphics APIs incorporated compute shaders that share resources and synchronization mechanisms with rendering commands.

A computational kernel is divided into work groups whose members can exchange data through on-chip storage and coordinate at defined synchronization points. Communication between different groups is more restricted because groups may execute in any order and may not reside on the processor simultaneously. Algorithms therefore tend to divide computation into phases separated by dispatch boundaries or device-wide synchronization operations.

GPU throughput is most effectively characterized by the interaction among arithmetic rate, memory traffic, occupancy, and control coherence. Arithmetic rate describes how many operations the execution units can issue under suitable conditions. Memory traffic determines whether operands can reach those units at a sufficient rate. Occupancy measures the amount of active work available to hide latency, while control coherence determines how fully the execution lanes participate in each instruction.

Programming and compilation

Graphics APIs such as Direct3D, Vulkan, and OpenGL provide interfaces through which applications create resources, compile shaders, and submit command streams. The CPU records commands that describe rendering or computation, and a driver translates those commands into packets interpreted by the device. Explicit APIs expose more responsibility for synchronization and memory placement, whereas older interfaces assign a larger portion of state tracking to the driver.

Shader programs are commonly written in a high-level shading language and compiled through one or more intermediate representations. The compiler maps scalar or vector operations onto the target instruction set, assigns registers, and arranges memory accesses. Register demand affects how many thread groups can remain resident at once, so a transformation that reduces the instruction count can still lower throughput if it substantially increases per-thread storage requirements.

Numerical behavior differs from that of many CPU environments. GPUs frequently support fused arithmetic operations and reduced-precision formats intended for high-throughput workloads. Floating-point results remain governed by the properties of floating-point arithmetic, including rounding and limited representation, but operation ordering may differ when thousands of threads contribute to a reduction. Reproducibility therefore depends on both the algorithm and the execution model rather than on nominal precision alone.

Specialized processing blocks

Modern graphics processors contain units whose operation is narrower than that of a programmable shader core. Video engines decode or encode compressed streams without expressing each stage as a shader program. Matrix-oriented units perform structured multiply-accumulate operations used in neural-network training and inference. Ray-intersection units accelerate recurring tests between rays and spatial data structures used by ray tracing.

These blocks do not replace the general execution array. They perform constrained portions of a workload, while shaders prepare data, manage exceptional cases, and combine intermediate results. Their presence extends the heterogeneous organization already established by texture samplers, rasterizers, and display controllers.

Performance and energy characteristics

Peak operation counts describe a theoretical issue rate and do not by themselves determine application performance. A workload with low arithmetic intensity can be limited by memory bandwidth, while a workload with irregular branches can leave execution lanes inactive. Small workloads may also fail to provide enough independent groups to occupy the available hardware.

Energy consumption is divided among arithmetic, data movement, control logic, and external memory activity. Moving an operand through the memory hierarchy can require more energy than applying a simple arithmetic operation to a value already held near an execution unit. GPU designs therefore use caches, compressed surface formats, and data-reuse mechanisms to reduce transfers, although their effectiveness depends on access patterns.

Thermal and electrical limits constrain sustained operating frequency. Devices dynamically select voltage and frequency states according to workload, temperature, and power limits. Consequently, performance measured over a short interval can differ from sustained performance even when the submitted command stream remains unchanged.