~1 min read · Updated Sep 6, 2026
Written & researched by Dr. Shahin Siami
Related Articles
Common Misconceptions About Parallel Computing and the Book's Final Lessons
After covering everything from thread-level parallelism to warehouse-scale computing, it is worth correcting persistent misconceptions about parallel systems that even experienced engineers sometimes hold. This article addresses common fallacies about scaling and parallel hardware, then closes out the parallel processing chapter by tying together the full journey from a single instruction to a building full of cooperating machines.
Real Stuff: Benchmarking CPUs Against GPUs and Multiprocessor Matrix Multiply
Comparing a CPU and a GPU fairly requires a model that accounts for both computational throughput and memory bandwidth limits together. This article introduces the roofline model used to compare real hardware like the Intel Core i7 and NVIDIA Tesla GPU, then shows how matrix multiplication is accelerated across multiple processors as the final practical application of this chapter's parallel concepts.
Benchmarking Multiprocessors and Modeling Parallel Performance
Measuring the performance of a parallel system requires different tools and metrics than measuring a single-core processor. This article covers the specialized benchmarks used to evaluate multiprocessor systems, explains how to model scaling behavior as more processors are added, and revisits Amdahl's Law in the context of real-world performance measurement.
Cluster Networking: Connecting to the World Outside
A cluster of machines is only useful if it can communicate efficiently both internally and with the outside world. This article covers the networking layers involved in cluster communication, the tradeoffs between latency and bandwidth at scale, and how clusters connect to external networks and users.
Clusters, Warehouse-Scale Computers, and Network Topologies
Beyond a single chip, parallelism extends to entire buildings full of independent computers working together. This article explains the shift from shared memory multiprocessing to clusters of separate machines, introduces the concept of warehouse-scale computing, and covers the network topologies that connect these independent machines efficiently.
An Introduction to GPUs: Massive Parallelism for Data-Heavy Workloads
A GPU takes the SIMD idea covered earlier in this series to an extreme scale, running thousands of lightweight threads simultaneously to process massive amounts of independent data. This article explains why GPUs are architecturally so different from CPUs, how their thread execution model works, and what kinds of workloads benefit most from this design.