Instruction-Level Parallelism: Executing More Than One Instruction at Once

The Limit of a Basic Pipeline

The five-stage pipeline covered earlier in this series can, at best, complete one instruction per clock cycle once it is full. This is a substantial improvement over the single-cycle design, but it still represents a hard ceiling: no matter how well hazards are managed, a single pipeline cannot exceed this one-instruction-per-cycle rate.

Going Beyond One Instruction Per Cycle

Instruction-Level Parallelism (ILP) refers to techniques that allow a processor to execute more than one instruction during the same clock cycle, by duplicating hardware resources and issuing multiple instructions into the pipeline simultaneously rather than one at a time.

Multiple Issue: Duplicating the Pipeline's Front End

A processor capable of this is called a Multiple-Issue or Superscalar processor. To support issuing more than one instruction per cycle, hardware duplicates key resources: multiple ALUs allow more than one arithmetic operation to execute simultaneously, additional read and write ports on the register file allow more operands to be accessed at once, and the instruction fetch stage must retrieve more than one instruction from memory per cycle.

Single-issue pipeline: 1 instruction per stage per cycle
Dual-issue pipeline:   up to 2 instructions per stage per cycle,
                       requiring 2 ALUs, extra register file ports,
                       and wider instruction fetch

Static Versus Dynamic Scheduling

Deciding which instructions can safely execute together in the same cycle can happen in two different ways. Static Scheduling is performed by the compiler ahead of time, arranging instructions in an order known to avoid conflicts before the program ever runs. Dynamic Scheduling is performed by hardware at runtime, examining instructions as they arrive and deciding on the fly which ones can be issued together, at the cost of additional hardware complexity to make these decisions quickly.

Why Dependencies Limit How Much Parallelism Is Possible

Not every pair of instructions can be issued together, no matter how much hardware is duplicated. If one instruction depends on the result of another, as covered earlier in this series when discussing data hazards, they cannot execute simultaneously regardless of how many ALUs are available; the dependent instruction must still wait for its input to become ready.

This means the actual performance benefit of multiple issue depends heavily on how much independent, parallelizable work exists in a given program. Code with many independent instructions benefits significantly, while code with a long chain of dependent instructions sees little improvement no matter how many issue slots are added.

Why This Approach Has Practical Limits

Continuing to add more issue slots and duplicate hardware resources produces diminishing returns past a certain point, since real programs rarely contain enough independent, ready-to-execute instructions to keep an ever-wider pipeline consistently full. This practical limit is part of why processor designers eventually turned toward multicore designs, discussed earlier in this series, as an additional way to increase overall throughput.