Why GPUs Look So Different From CPUs
A CPU, as covered throughout most of this series, is optimized to run a small number of complex, independent instruction streams as fast as possible, using techniques like out-of-order execution and branch prediction discussed earlier. A GPU (Graphics Processing Unit) takes an almost opposite approach: it sacrifices the sophistication of any single execution stream in exchange for running an enormous number of simple, simultaneous threads.
The Origin: Graphics Rendering as Massive Data Parallelism
GPUs were originally designed for rendering graphics, a workload that requires applying the same relatively simple calculation, such as computing a pixel's color, to millions of independent pixels simultaneously. This is an extreme, large-scale version of the data parallelism discussed earlier in this series regarding SIMD and subword parallelism, and it shaped the GPU's entire architecture around running the same operation across enormous amounts of independent data at once.
The GPU Execution Model: Threads Organized in Groups
Rather than a handful of complex cores, a GPU contains a very large number of simple processing units, organized so that groups of threads execute the exact same instruction together in lockstep, a model closely related to the SIMD and SPMD concepts discussed earlier in this series but implemented at a much larger scale.
CPU approach: few complex cores,
each running a different, independent instruction stream
GPU approach: thousands of simple cores,
large groups executing the same instruction
simultaneously on different dataWhy This Design Trades Flexibility for Throughput
Because groups of GPU threads execute in lockstep, a GPU performs best when every thread in a group follows the exact same execution path. If threads within the same group need to take different branches, some threads must wait idle while others execute, a situation called Divergence, which reduces the efficiency advantage that made GPUs attractive in the first place.
What Kinds of Workloads Benefit Most
GPUs excel at workloads with abundant, independent data parallelism and minimal branching divergence between elements, such as graphics rendering, large-scale matrix operations discussed earlier in this series regarding matrix multiplication, and machine learning training, all of which apply the same relatively simple mathematical operation across enormous datasets. Workloads dominated by complex, unpredictable branching or long chains of dependent sequential operations generally perform poorly on GPU hardware and are better suited to a CPU.
Why GPUs Have Become Central to Modern Computing
The explosive growth of machine learning, which relies heavily on massive matrix and vector operations, has made GPUs central to modern computing far beyond their original graphics rendering purpose, turning what began as a specialized graphics accelerator into one of the most important classes of computing hardware in use today.