An Overview of Pipelining: Overlapping Instruction Execution

The Core Weakness of a Single-Cycle Design

In the single-cycle implementation covered earlier in this series, every instruction takes exactly one clock cycle, but that cycle must be long enough to accommodate the slowest instruction the processor supports, typically a load instruction that touches memory. This means simple instructions like addition sit idle for much of the cycle, wasting available hardware capacity.

The Assembly-Line Analogy

Pipelining solves this by overlapping the execution of multiple instructions, much like an assembly line where different workers handle different stages of production simultaneously on different products. While one instruction is being decoded, another can already be fetched, and a third can be executing — all in the same clock cycle, just at different stages.

The Standard Five-Stage Pipeline

A classic RISC pipeline, including the one used for RISC-V in this chapter, divides instruction execution into five distinct stages:

• IF (Instruction Fetch): read the next instruction from instruction memory.
• ID (Instruction Decode): decode the instruction and read the required values from the register file.
• EX (Execute): perform the arithmetic operation or calculate a memory address using the ALU.
• MEM (Memory Access): read from or write to data memory, relevant only for load and store instructions.
• WB (Write Back): write the final result back into the register file.

Each stage is handled by a distinct piece of hardware, and a set of pipeline registers between stages hold the intermediate results of one instruction while the next instruction moves into the earlier stage.

Visualizing Overlapped Execution

Cycle:      1    2    3    4    5    6    7
Instr 1:   IF   ID   EX   MEM  WB
Instr 2:        IF   ID   EX   MEM  WB
Instr 3:             IF   ID   EX   MEM  WB

At cycle 3 in this diagram, three different instructions are being processed simultaneously, each in a different stage — this overlap is the entire source of pipelining's performance benefit.

Throughput Improves, But Individual Latency Does Not

It is a common misconception that pipelining makes each instruction execute faster. In reality, a single instruction still takes the same total time, or slightly longer, to move through all five stages. What improves is Throughput: once the pipeline is full, a new instruction can complete roughly every single clock cycle instead of waiting for the entire multi-stage process to finish before starting the next one.

Why This Idea Needs Careful Handling

Overlapping instructions this way introduces new problems that a single-cycle design never had to face — instructions that depend on each other's results, and branches that are not yet resolved when the next instruction needs to be fetched. These complications, known as pipeline hazards, are the focus of the sections that follow later in this series.