GPU Architecture

A graphics processing unit, or GPU, is a processor built to run many similar operations in parallel. Its architecture combines groups of execution lanes, fast on-chip memory, caches, and specialized graphics or matrix hardware to favor throughput over fast completion of one instruction stream.

itComputer architecture and hardware

GPU Architecture

A graphics processing unit, or GPU, is a throughput processor. It keeps many arithmetic operations in flight so the chip can process large collections of pixels, vertices, matrix elements, particles, or other independent data. A central processing unit, or CPU, usually devotes more hardware to reducing the delay of a few instruction streams. A GPU devotes more hardware to parallel execution and accepts that one item may wait while many other items make progress.

That distinction describes a design priority, not a rule that GPUs are always faster. A workload needs enough parallel work, suitable memory access, and enough computation to repay data movement and launch overhead. Serial control flow, small jobs, irregular pointer chasing, and frequent host-device transfers can leave a GPU underused.

The execution hierarchy

GPU software describes a large set of lightweight work items. CUDA calls them threads. OpenCL and related models use work-items. Hardware collects neighboring work items into an execution group that shares an instruction stream. NVIDIA calls that group a warp; AMD calls it a wavefront. The exact width is architecture-specific, so portable software should obtain it from the programming model or device rather than assume one vendor's width.

A kernel is a function launched across many work items. Software groups the work items into blocks or work-groups. A GPU schedules each block onto a compute unit, such as an NVIDIA streaming multiprocessor, an AMD compute unit, or an Intel Xe-core. Work items in one block can cooperate through fast local storage and block-scoped synchronization. Separate blocks must remain independently schedulable unless a programming model supplies a wider coordination mechanism.

The hierarchy is therefore:

application on host
        ↓ submits work
kernel grid or dispatch
        ↓ divided into
blocks or work-groups
        ↓ divided into
warps, wavefronts, or subgroups
        ↓ issue across
execution lanes and specialized units

This arrangement lets the same architecture scale. A larger GPU runs more blocks at once; a smaller GPU runs the same blocks in more waves. Correct code does not depend on which block starts first.

GPU Architecture

GPU Architecture

The execution hierarchy

What happens inside a compute unit

Relevant careers