openskills.info
Course catalog
Course Preview

Memory Hierarchy

Memory hierarchy arranges registers, caches, main memory, and storage in layers with different speed, capacity, and cost. It keeps frequently used data near the processor while larger, slower layers retain the rest.

itComputer architecture and hardware

Memory Hierarchy

A memory hierarchy is the layered path between a processor and the data or instructions it needs. The upper layers are small and fast. Lower layers hold more data but take longer to reach. A typical path begins with registers, continues through one or more processor caches, reaches dynamic random-access memory, and may extend through virtual memory to persistent storage.

No single storage technology provides the lowest latency, largest capacity, lowest cost, and persistence at the same time. A hierarchy combines technologies with different strengths. It works because programs usually reuse a limited working set during a given interval.

The access path

A load or instruction fetch starts with an address. The processor first checks the closest applicable storage. A register operand is already available to an execution unit. Other addresses normally pass through an L1 cache, then larger downstream caches, and finally main memory. If virtual memory must bring an absent page from storage, the operating system joins the path.

Each cache holds copies of blocks from the next level. A cache hit finds the requested block at the checked level. A cache miss sends the request farther down the hierarchy. The returned block fills a cache line so later accesses can hit closer to the processor.

Two forms of locality make those copies useful:

  • Temporal locality means a recently accessed location is likely to be accessed again soon.
  • Spatial locality means an access makes nearby locations more likely to be accessed soon.

A loop that repeatedly reads a small array has temporal locality. Sequentially scanning the array has spatial locality. A pointer-chasing workload spread across memory may exhibit little spatial locality because adjacent bytes are not the next useful data.

Layers and managers

Continue the course

This section is part of the paid course.

See pricing to subscribe, or log in if you already have access.

Where this skill leads

Relevant careers

See how this topic contributes to broader role-level skill maps.

Sources