Infrastructure, Systems, and Connectivity
No single memory technology is simultaneously fast, large, and cheap. This article introduces the concept of a memory hierarchy that combines several different memory technologies to approximate the speed of the fastest one at the cost of the cheapest, then walks through the core technologies that make up each level.
A cache works because programs tend to access the same or nearby data repeatedly rather than randomly. This article explains the principle of locality that makes caching effective, how a direct-mapped cache locates data using an address, and what happens on a cache hit versus a cache miss.
Not all cache misses are the same, and understanding their causes is the first step toward improving performance. This article covers how to calculate the real performance impact of caching using miss rate and miss penalty, classifies the three common causes of cache misses, and explains practical strategies for reducing each type.
Memory hardware is not perfectly reliable; electrical noise and physical defects can silently flip stored bits. This article explains how error detection and correction codes let hardware notice, and in many cases automatically fix, these corrupted values before they cause incorrect program behavior.
A single physical computer can appear to run several completely separate operating systems at once, each unaware of the others' existence. This article explains what a virtual machine actually is, how a hypervisor manages this illusion, and why this technology matters for both server consolidation and system security.
Programs behave as if they have access to a huge, private block of memory, even though physical RAM is limited and shared among many running processes. This article explains how virtual memory creates this illusion through address translation, how page tables and the TLB make translation fast, and what happens when needed data is not currently in physical memory.
Caches and virtual memory appear at first glance to be very different systems, yet both are answering the exact same four fundamental questions. This article shows how those four questions unify block placement, block identification, block replacement, and write handling across every level of the memory hierarchy, from tiny caches to disk-backed virtual memory.
A cache does not just store data passively; hardware control logic must sequence through several distinct steps to handle a miss correctly. This article explains how a finite-state machine models this control logic, walks through the states involved in handling a cache hit and a cache miss, and shows why this formal model makes cache controller design easier to reason about.
When multiple processor cores each have their own private cache but share the same underlying memory, a serious correctness problem emerges: different cores can end up with conflicting views of the same memory location. This article explains what cache coherence means, why it becomes necessary in multicore systems, and how snooping-based protocols keep every core's cached data consistent.
A single storage drive can fail at any time, potentially destroying valuable data with no warning. This article explains how RAID combines multiple disks to improve reliability, performance, or both, covering the tradeoffs between mirroring, striping, and parity-based approaches.
Theoretical cache and memory concepts take concrete form in real commercial processors with multiple cache levels and carefully tuned sizes. This article compares real memory hierarchies in ARM and Intel processors, shows how cache blocking accelerates matrix multiplication in practice, and closes out the memory hierarchy chapter by addressing common misconceptions.
Writing software that correctly and efficiently uses multiple processors is fundamentally harder than writing sequential code. This article explains why parallel programming introduces unique challenges around coordination and diminishing returns, then covers the classic Flynn's taxonomy that categorizes parallel hardware into SISD, MIMD, SIMD, and related models.
