Managing Static (Leakage) Power - Computer Architecture Techniques for Power-Efficiency - page 161

and every cache line, la cache decay, veers off the desired course of simplicity. Instead, Petit et
al. propose simpler heuristics to filter the lines that are put in the drowsy mode.
Considering the cache lines in an associative set, it is obvious that if there is a live cache
line among them it must be the most recently used (MRU) line. Thus, the first policy is to
exclude the MRU line from going into drowsy mode. In fact, the policy allows only the MRU
line to remain awake in the set. The policy is called MRU ON, or simply MRO. Upon a hit
on a drowsy line, the line is woken up and becomes the MRU line. The previous MRU line is
put in the drowsy mode. According to Petit et al., 92% of the cache hits are hits on the MRU
line so a change in the MRU status, which incurs dynamic switching, is rare.
The second policy simply keeps awake the two most recently used lines per set inasmuch
as both are good candidates for being active (alive) rather than idle (dead). The policy is called
Two-MRO (TMRO). This choice is justified because most of the remaining 8% of the hits that
do not hit the MRU line are accesses to the second most recently used line. Since more lines per
set are kept awake energy savings are less than MRO but accesses to drowsy lines are minimized.
Both these policies try to minimize the performance penalty from accessing drowsy lines.
This penalty, however, is only important if the transition from drowsy to active takes a few
cycles. Petit et al. examine these two policies with a transition latency of three cycles which is
reasonable (if not overly aggressive) for a number of designs. 14
Petit et al. conduct experiments which show these two simple policies, MRO and TMRO,
offer power savings of about 72% and 48%, with a hit ratio on active lines of 92% and 98%,
respectively. However, neither can adapt to the dynamic access behavior of various applications.
For this reason, the authors propose an adaptive variation that dynamically selects, per set, one
of the two policies depending on the number of accessed lines in the set within a time window.
This adaptive policy called Reused-MRO (RMRO), settles midway between the MRO and
TMRO and outperforms the Simple policy of Flautner et al. both in leakage savings (65%
versus 62%) and in the hit ratio on active lines ( ∼ 99%).

5.3.2 Drowsy Instruction Caches
One situation where the Flautner et al. Simple policy—periodically putting the whole cache
in drowsy mode—does not work very well is the instruction cache [ 138 ]. This is because
instructions exhibit strong temporal and spacial locality and any delay in fetch shows up
immediately on performance. Putting active instructions in drowsy mode is a sure way to lower

14 The wake-up latency depends on how big is the cache line (number of cells, length of power lines) but more
importantly how large is the voltage controller. To drop the latency to the overly aggressive 1 or 2 cycles in the
initial proposal [ 77 ], a quite large controller (128 to 256 L min wide) must be used, significantly increasing the
overhead per cache line [ 137 ].