Instruction-Level Parallelism and Its Exploitation - Computer Architecture: A Quantitative Approach - page 208

plexity of doing this for a larger number of instructions per clock becomes a chief limitation in
the issue width.
How Much to Speculate
One of the significant advantages of speculation is its ability to uncover events that would oth-
erwise stall the pipeline early, such as cache misses. This potential advantage, however, comes
with a significant potential disadvantage. Speculation is not free. It takes time and energy, and
the recovery of incorrect speculation further reduces performance. In addition, to support the
higher instruction execution rate needed to benefit from speculation, the processor must have
additional resources, which take silicon area and power. Finally, if speculation causes an ex-
ceptional event to occur, such as a cache or translation lookaside buffer (TLB) miss, the poten-
tial for significant performance loss increases, if that event would not have occurred without
speculation.
To maintain most of the advantage, while minimizing the disadvantages, most pipelines
with speculation will allow only low-cost exceptional events (such as a first-level cache miss)
to be handled in speculative mode. If an expensive exceptional event occurs, such as a second-
level cache miss or a TLB miss, the processor will wait until the instruction causing the event
is no longer speculative before handling the event. Although this may slightly degrade the
performance of some programs, it avoids significant performance losses in others, especially
those that suffer from a high frequency of such events coupled with less-than-excellent branch
prediction.
In the 1990s, the potential downsides of speculation were less obvious. As processors have
evolved, the real costs of speculation have become more apparent, and the limitations of wider
issue and speculation have been obvious. We return to this issue shortly.
Speculating through Multiple Branches
In the examples we have considered in this chapter, it has been possible to resolve a branch be-
fore having to speculate on another. Three different situations can benefit from speculating on
multiple branches simultaneously: (1) a very high branch frequency, (2) significant clustering
of branches, and (3) long delays in functional units. In the first two cases, achieving high per-
formance may mean that multiple branches are speculated, and it may even mean handling
more than one branch per clock. Database programs, and other less structured integer com-
putations, often exhibit these properties, making speculation on multiple branches important.
Likewise, long delays in functional units can raise the importance of speculating on multiple
branches as a way to avoid stalls from the longer pipeline delays.
Speculating on multiple branches slightly complicates the process of speculation recovery
but is straightforward otherwise. As of 2011, no processor has yet combined full speculation
with resolving multiple branches per cycle, and it is unlikely that the costs of doing so would
be justified in terms of performance versus complexity and power.
Speculation and the Challenge of Energy Efficiency
What is the impact of speculation on energy efficiency? At first glance, one might argue that
using speculation always decreases energy efficiency, since whenever speculation is wrong it
consumes excess energy in two ways:
1. The instructions that were speculated and whose results were not needed generated excess
work for the processor, wasting energy.