Infrastructure, Systems, and Connectivity
A single processor core often sits idle waiting for slow memory operations to complete, wasting valuable execution resources. This article explains how hardware multithreading lets a core rapidly switch between multiple independent instruction streams to fill those idle moments, covering both fine-grained and simultaneous multithreading approaches.
Multicore processors are the most common form of parallel hardware today, but the way their cores actually share memory varies in important ways. This article explains the shared memory multiprocessor model, contrasts uniform and non-uniform memory access designs, and covers how the operating system and programmer coordinate work across cores.
A GPU takes the SIMD idea covered earlier in this series to an extreme scale, running thousands of lightweight threads simultaneously to process massive amounts of independent data. This article explains why GPUs are architecturally so different from CPUs, how their thread execution model works, and what kinds of workloads benefit most from this design.
Beyond a single chip, parallelism extends to entire buildings full of independent computers working together. This article explains the shift from shared memory multiprocessing to clusters of separate machines, introduces the concept of warehouse-scale computing, and covers the network topologies that connect these independent machines efficiently.
A cluster of machines is only useful if it can communicate efficiently both internally and with the outside world. This article covers the networking layers involved in cluster communication, the tradeoffs between latency and bandwidth at scale, and how clusters connect to external networks and users.
Measuring the performance of a parallel system requires different tools and metrics than measuring a single-core processor. This article covers the specialized benchmarks used to evaluate multiprocessor systems, explains how to model scaling behavior as more processors are added, and revisits Amdahl's Law in the context of real-world performance measurement.
Comparing a CPU and a GPU fairly requires a model that accounts for both computational throughput and memory bandwidth limits together. This article introduces the roofline model used to compare real hardware like the Intel Core i7 and NVIDIA Tesla GPU, then shows how matrix multiplication is accelerated across multiple processors as the final practical application of this chapter's parallel concepts.
After covering everything from thread-level parallelism to warehouse-scale computing, it is worth correcting persistent misconceptions about parallel systems that even experienced engineers sometimes hold. This article addresses common fallacies about scaling and parallel hardware, then closes out the parallel processing chapter by tying together the full journey from a single instruction to a building full of cooperating machines.
Relational databases organize data into structured tables that can be queried, updated, and managed using SQL, a language designed specifically for working with structured data. This article introduces what a relational database actually is, walks through writing a first SQL query, covers the basic query clauses every database user relies on, and explains the fundamentals of creating and managing tables and their data.
The real power of a relational database emerges when data is split across multiple related tables instead of being duplicated everywhere. This article explains why splitting data across tables avoids redundancy, covers the foreign key relationship that connects tables together, walks through the different types of JOIN used to query across related tables, and introduces a few more SQL techniques for managing table structure and data safely.
Writing SQL queries is only half the picture; designing a database well before writing any queries at all determines whether that database will remain reliable, efficient, and maintainable as an application grows. This article explains the core goals every database design should pursue, walks through the overall design process from requirements to implementation, and introduces the key phases every well-designed database passes through.
Before a single table is created, conceptual design requires identifying which real-world things a database needs to represent and what details about each one actually matter. This article explains what qualifies as an entity, how to identify the attributes that describe it, the different types of attributes that appear in practice, and how choosing an appropriate identifying key shapes the rest of the design.
