Why One Memory Type Is Never Enough
An ideal memory would be extremely fast, hold an enormous amount of data, and cost very little. In practice, no single technology achieves all three at once: faster memory is always more expensive per byte and typically smaller, while cheaper, larger memory is always slower. This tradeoff is the central problem that the Memory Hierarchy is designed to solve.
The Core Idea: Combine Several Levels
Rather than choosing one memory technology, a computer combines several different levels of memory, each with a different balance of speed, size, and cost. Data that is frequently used is kept in the smallest, fastest, most expensive level, while data used less often resides in progressively larger, slower, and cheaper levels further away from the processor.
Fastest, smallest, most expensive:
Registers → Cache → Main Memory → Disk/SSD
Slowest, largest, cheapestSRAM: The Foundation for Fast, Small Memory
SRAM (Static Random Access Memory) stores each bit using a small circuit of transistors that holds its value continuously as long as power is supplied, without needing to be refreshed periodically. This makes SRAM very fast to access, but each storage cell requires several transistors, making it physically larger and more expensive per bit than the alternative used for main memory.
DRAM: The Foundation for Large, Cheap Memory
DRAM (Dynamic Random Access Memory) stores each bit using a single transistor and a capacitor, which is far more compact and cheaper to manufacture per bit than SRAM. The tradeoff is that the capacitor's charge leaks over time, so DRAM must be periodically Refreshed — its stored values re-written at regular intervals — to avoid losing data, and reading from it takes noticeably longer than reading from SRAM.
Why These Two Technologies Are Used Differently
Because of these tradeoffs, SRAM is used almost exclusively for small, extremely fast Cache memory built directly into or very close to the processor, while DRAM is used for larger Main Memory that holds far more data at a fraction of the per-bit cost, at the price of slower access time.
Beyond SRAM and DRAM: Persistent Storage
Below main memory in the hierarchy sits persistent storage, such as Solid-State Drives (SSDs) or traditional magnetic Hard Disk Drives (HDDs), which retain data even when power is removed, hold vastly more data than DRAM, but are significantly slower to access, particularly hard disk drives, which involve mechanical movement to physically locate data.
Why Understanding This Foundation Matters
Every technique discussed in the remainder of this chapter — caching strategies, virtual memory, and performance optimization — exists specifically because of the fundamental tradeoffs described here between speed, size, and cost. Without appreciating why no single memory technology can satisfy every requirement, the design decisions behind cache hierarchies would appear arbitrary rather than necessary.