Virtual Memory
Virtual memory gives each process a private address space and maps its addresses to physical memory or other backing storage. The operating system and processor cooperate to provide isolation, controlled sharing, and demand-loaded pages.
itOperating systems | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic — Virtual Memory
Virtual memory is the arrangement that lets a program use its own stable addresses while the processor and operating system decide where the data lives. It exists because handing every program a map of physical RAM would make sharing, protection, and moving data about roughly as relaxing as organizing a library during an earthquake.
The important split is between a virtual address, which a program uses, and a physical page frame, which is where a page happens to be resident. The MMU translates one into the other through page tables, with a TLB keeping recent answers nearby. The address can remain useful while its backing page is copied, discarded, loaded from a file, or restored from swap. Stability above; activity below.
A page fault is the phrase most likely to sound like a fire alarm while doing ordinary work. It is a control transfer to the kernel because the current mapping cannot finish the access. First use of anonymous memory can allocate a zero-filled page. A file mapping can fetch a page. A copy-on-write write can make a private copy. The bad version is an access the mapping or permissions cannot authorize.
The second trap is asking, “How much memory does this process use?” as though the machine had prepared one polite answer. Virtual size describes mapped or reserved address space. RSS describes pages resident for a process. PSS divides shared resident pages among their users. Commit describes promised writable backing. Each number is correct; none is a complete biography.
When physical memory tightens, the kernel reclaims pages outside the active working set. Clean file-backed pages can leave and return from their file. Anonymous pages may use swap when it is available. If the same pages are evicted and faulted back repeatedly, the system is thrashing: it has turned useful work into an elaborate courier service for its own data.
Read the intro for the translation path and backing choices. Use the slides when the distinctions between a TLB miss, page fault, and reclaim path need a compact map. Keep the cheatsheet nearby for the accounting terms and diagnostic rules. Field Notes adds the operational signals that tell you when a technically valid arrangement has become a performance problem.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://docs.kernel.org/admin-guide/mm/concepts.html
Supports
- Virtual addresses, physical pages, page tables, demand paging, protection, and sharing
- Anonymous memory, page cache, reclaim, compaction, swap, and OOM concepts
- https://docs.kernel.org/mm/page_tables.html
Supports
- Hierarchical page tables, page frames, large mappings, MMU, TLB, dirty and permission bits
- Expected faults, copy-on-write, invalid accesses, SIGSEGV, and Linux page-table evolution
- Atlas and Intel 80386 virtual-memory milestones
- https://learn.microsoft.com/en-us/windows/win32/memory/virtual-address-space
Supports
- Private process address spaces and page-table translation
- https://learn.microsoft.com/en-us/windows/win32/memory/virtual-address-space-and-physical-storage
Supports
- Working sets, page organization, paging-file backing, and transparent movement of pages
- https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/managing-memory-sections
Supports
- Demand allocation, first-access faults, file-backed loading, and copy-on-write views
- https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/file-backed-and-page-file-backed-sections
Supports
- File-backed, page-file-backed, shared, and copy-on-write section behavior
- https://www.man7.org/linux/man-pages/man2/mmap.2.html
Supports
- Linux anonymous and file mappings, shared and private mappings, protections, and fixed-address hazards
- https://www.man7.org/linux/man-pages/man5/proc_pid_smaps.5.html
Supports
- RSS, PSS, clean and dirty private or shared pages, anonymous memory, swap, page sizes, and flags
- https://www.man7.org/linux/man-pages/man5/proc_pid_statm.5.html
Supports
- Virtual size, resident and shared counts, counter accuracy, and the detailed smaps alternative
- https://www.man7.org/linux/man-pages/man5/proc_pid_stat.5.html
Supports
- Process virtual-memory size and resident-set definitions
- https://docs.kernel.org/mm/overcommit-accounting.html
Supports
- Linux overcommit modes, commit limits, mapping costs, and allocation guarantees
- https://docs.kernel.org/admin-guide/sysctl/vm.html
Supports
- OOM selection, overcommit controls, swap readahead, and memory-policy behavior
- https://docs.kernel.org/admin-guide/mm/transhuge.html
Supports
- Transparent huge-page purpose, fallback, promotion, demotion, and splitting
- https://docs.kernel.org/admin-guide/mm/multigen_lru.html
Supports
- Working-set reclaim, refault protection, thrashing prevention, and OOM trade-offs
- https://docs.kernel.org/admin-guide/cgroup-v2.html
Supports
- Memory-control-group limits, reclaim, pressure, and OOM events
- https://docs.kernel.org/admin-guide/mm/zswap.html
Supports
- Compressed swap cache, swap input and output trade-offs, and backing-device eviction
- https://www.cs.manchester.ac.uk/about/history-and-heritage/
Supports
- 1962 Atlas operation and first use of virtual memory
- https://www.vm.ibm.com/history/timeline.html
Supports
- CP-40 in 1964, CP-67 in 1967, and VM/370 in 1972
- https://multicians.org/history.html
Supports
- June 1967 Multics paging and segmentation development milestone
- Multics three-level virtual-memory backing model
- https://web.mit.edu/Saltzer/www/publications/f7y/f7y.html
Supports
- Multics paging and segmentation design and project history
- https://www.ibm.com/history/system-370
Supports
- August 1972 System/370 virtual-memory support and practical effect
- https://www.ibm.com/docs/en/zos-basic-skills?topic=storage-brief-history-virtual-64-bit-addressability
Supports
- System/370 virtual storage and 1983 System/370-XA 31-bit addressability
- https://timeline.intel.com/1985/raising-the-bar-with-the-386
Supports
- Intel 80386 introduction on October 17, 1985
- https://ir.amd.com/financial-information/sec-filings/content/0001193125-04-037179/d10k.htm
Supports
- April 2003 AMD Opteron introduction and AMD64 extension of x86 to 64-bit computing
- https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/40332.pdf
Supports
- AMD64 long mode, expanded virtual addressing, and large-memory motivation
- https://kernel.org/doc/ols/2011/ols2011-masters.pdf
Supports
- Transparent Huge Pages included in Linux 2.6.38 in March 2011
- https://lkml.iu.edu/hypermail/linux/kernel/1709.0/01229.html
Supports
- Linux 4.14 as the first kernel able to enable five-level x86 paging in 2017
- https://github.com/sindresorhus/awesome
Supports
- Discovery path to the Awesome eBPF list
- https://github.com/qmonnet/awesome-ebpf
Supports
- Ecosystem discovery for BCC and bpftrace
- https://github.com/awesome-foss/awesome-sysadmin
Supports
- Ecosystem discovery for Sysdig as a Linux troubleshooting tool
- https://github.com/iovisor/bcc
Supports
- BCC memory, process, page-cache, reclaim, compaction, OOM, and allocation tools
- https://bpftrace.org/one-liners
Supports
- bpftrace page-fault counting and stack profiling examples
- https://docs.sysdig.com/en/docs/sysdig-monitor/using-monitor/metrics/metrics-library/sysdig-legacy-format/system/
Supports
- Process virtual memory, resident memory, swap, and major or minor page-fault metrics
- https://engineering.fb.com/2022/06/20/data-infrastructure/transparent-memory-offloading-more-memory-at-a-fraction-of-the-cost-and-power/
Supports
- Production use of PSI rather than page-fault counts alone to regulate memory reclaim
- Failure mode of lowering a cgroup limit as a reclaim mechanism
- https://docs.kernel.org/accounting/psi.html
Supports
- PSI memory stall definitions and the distinction between some and full pressure
- Memory-pressure signals for sizing, load shedding, and avoiding OOM disruption
