CPU Architecture

A central processing unit, or CPU, executes the machine instructions that make software run. CPU architecture gives you a working model of that process. It connects an instruction set, a processor core, memory, and the operating system.

You do not need to design silicon to use this model. It helps you read performance data, understand assembly, choose hardware, and explain why two compatible processors behave differently.

Architecture is a contract

An instruction set architecture, or ISA, defines the behavior software can observe. It specifies instructions, registers, data types, control flow, memory behavior, and events such as exceptions.

A microarchitecture is a particular implementation of an ISA. It defines internal choices such as pipeline depth, execution units, cache organization, and instruction timing.

The distinction is practical:

The ISA says what a correct instruction does.
The microarchitecture decides how to perform that instruction.

Two processors can run the same machine code while using different internal designs. Arm documents Cortex-A53 and Cortex-A72 as different microarchitectures implementing the same architecture. RISC-V also separates its software-visible ISA from implementation details.

Follow one instruction

A simple processor appears to repeat a short cycle:

Fetch an instruction from the address in the program counter.
Decode the instruction into an operation and operands.
Read source registers or request data from memory.
Execute the operation.
Write the result.
Select the next instruction address.

This sequence explains the dependencies. It is not a literal schedule for every CPU. Modern cores overlap these steps across many instructions.

The program counter identifies an instruction address. A sequential instruction advances it. A branch, jump, call, return, exception, or interrupt can choose another address.

The decoder interprets encoded instruction fields. The fields can name an operation, source registers, a destination register, or an immediate value carried inside the instruction.

Registers hold values directly available to instructions. General-purpose registers commonly hold integers, addresses, and intermediate results. An ISA can define separate or extended register sets for floating-point, vector, status, or control information.

The pipeline overlaps work

A pipeline divides instruction processing into stages. While one instruction executes, a later instruction can decode and another can fetch. This overlap can increase throughput without making one instruction pass through every stage instantly.

Dependencies can interrupt the flow. A data hazard occurs when an instruction needs a result that an earlier instruction has not produced. A control hazard occurs when the next instruction address depends on an unresolved branch. A structural hazard occurs when concurrent instructions need the same limited resource.

A core can respond by stalling, forwarding a result directly between stages, adding resources, or scheduling independent work. Each choice trades area, energy, complexity, and performance.

Pipeline depth alone does not measure quality. A deeper pipeline can support a shorter clock period, but it can increase the work discarded after a wrong branch prediction. Workload behavior and the complete design matter.

Front end, execution engine, and retirement

A useful modern-core map has three regions.

The front end finds and prepares instructions. It predicts control flow, fetches instruction bytes, decodes them, and supplies operations to the rest of the core.

The execution engine performs the work. It contains resources for integer arithmetic, floating-point or vector operations, branches, and loads and stores. Different operations can have different latency and throughput.

The retirement stage makes completed results architecturally visible in the required order. This boundary lets a core execute internally in a different order while preserving the behavior promised by the ISA.

An in-order core begins execution according to program order. An out-of-order core can issue a later independent operation while an earlier operation waits. Out-of-order execution looks for instruction-level parallelism inside one instruction stream.

A superscalar core can start more than one operation in a cycle when dependencies and resources allow it. Superscalar width is a capacity, not a guarantee. A dependency chain, cache miss, or limited execution port can leave capacity unused.

Prediction keeps the front end moving

A conditional branch creates two possible paths. Waiting for the condition would leave later pipeline stages idle. A branch predictor estimates the path and target so the core can fetch ahead.

Speculative execution performs work before the processor knows that an assumption is correct. Correctly predicted work can later retire. Work based on a wrong prediction must be squashed so it does not change architectural state.

Architectural state is not the whole security story. Transient work can affect microarchitectural state, including caches and predictors. Those effects can form side channels. Treat speculation as both a performance mechanism and a security boundary.

Memory dominates many workloads

Execution units can operate much faster than main memory can answer every request. CPUs therefore use a memory hierarchy.

A common path is:

registers -> level-one cache -> larger caches -> main memory

A cache stores copies of blocks from another memory level. A cache hit supplies a requested block from the checked level. A miss continues to another level and can delay dependent instructions.

Separate level-one instruction and data caches let the front end fetch instructions while the execution engine accesses data. Larger caches can be private to one core or shared by several cores. Exact arrangements are microarchitecture choices.

Programs benefit from temporal locality when they reuse data soon. They benefit from spatial locality when they access nearby addresses. Data layout and traversal order therefore affect performance without changing the algorithm's mathematical result.

A hardware prefetcher predicts future memory accesses and requests blocks early. Useful prefetching hides latency. Incorrect prefetching consumes cache capacity and memory bandwidth.

Translation adds another lookup

Applications normally issue virtual addresses. The processor and operating system translate them to physical addresses using page tables.

A translation lookaside buffer, or TLB, caches recent address translations. A TLB hit avoids a page-table walk. A TLB miss requires additional memory activity before the original access can complete.

Virtual memory provides separate address spaces, permissions, and flexible physical placement. The translation machinery is part of CPU architecture even though the operating system builds and manages the page tables.

More cores create more concurrency

A core executes an instruction stream. A CPU package can contain multiple cores. Some cores expose multiple hardware threads, allowing more than one architectural instruction stream to share core resources.

More cores improve throughput only when software has work that can run concurrently. A serial dependency remains serial. Shared memory also requires coordination.

Cache coherence coordinates cached copies of shared locations. A memory consistency model defines which orderings and values concurrent software may observe. Coherence does not turn a multi-instruction update into one atomic operation. Correct shared-memory software still needs synchronization.

RISC-V illustrates the separation clearly. Its RVWMO model permits some memory operations from another hardware thread to appear reordered. Fences and atomic instructions provide stronger ordering where software needs it.

Privilege protects the system

General-purpose ISAs distinguish application execution from privileged system control. Privileged software configures address translation, handles interrupts and exceptions, controls devices, and manages processor state that applications must not change directly.

An exception is associated with instruction execution, such as an unsupported instruction or a page fault. An interrupt is asynchronous to the current instruction stream, such as a timer or device request. Both can transfer control to a handler under architecture-defined rules.

Virtualization adds another control layer. Hardware support can let a hypervisor run guest operating systems while controlling their privileged operations and memory translations.

Measure the workload, not the label

Clock frequency states cycles per second. It does not state how much useful work finishes in each cycle.

CPU time depends on several interacting factors:

The instructions selected by the program and compiler
The work each instruction performs
Dependencies between instructions
Front-end and execution-resource limits
Branch-prediction accuracy
Cache and TLB behavior
Memory bandwidth and latency
Available parallel work
Power and thermal limits

Latency measures how long one operation takes. Throughput measures how many operations finish over time. Improving one does not always improve the other.

Use representative workloads and elapsed-time measurements first. Hardware performance counters can then help explain where cycles went. A model name, core count, cache size, or frequency alone cannot predict application performance.

When this map helps

Use CPU architecture knowledge when you:

Interpret compiler-generated assembly
Diagnose branch, cache, or TLB bottlenecks
Decide whether extra cores can help a workload
Explain why memory layout changes performance
Evaluate latency, throughput, energy, and cost tradeoffs
Understand operating-system, hypervisor, and security controls
Compare processors that implement the same or different ISAs

This course gives you an orientation. Deeper competence comes from tracing a simple pipeline, studying one ISA manual, measuring real programs, and then learning a specific microarchitecture.

Limits of the model

The front-end, execution, and retirement map hides many implementation details. Real CPUs can translate instructions into internal operations, rename registers, use several cache levels, divide cores into clusters, or combine different core designs in one package.

Vendor manuals also separate architectural guarantees from model-specific behavior. Confirm exact instruction semantics in an ISA reference. Confirm timing and tuning advice in documentation for the processor you actually use.