Review of Memory Hierarchy - Computer Architecture: A Quantitative Approach - page 573

As we can see from the figures, the compulsory miss rate of the SPEC2000 programs is very
small, as it is for many long-running programs.
Having identified the three C's, what can a computer designer do about them? Conceptu-
ally, conflicts are the easiest: Fully associative placement avoids all conflict misses. Full asso-
ciativity is expensive in hardware, however, and may slow the processor clock rate (see the
example on page B-29), leading to lower overall performance.
There is litle to be done about capacity except to enlarge the cache. If the upper-level
memory is much smaller than what is needed for a program, and a significant percentage of
the time is spent moving data between two levels in the hierarchy, the memory hierarchy is
said to thrash . Because so many replacements are required, thrashing means the computer runs
close to the speed of the lower-level memory, or maybe even slower because of the miss over-
head.
Another approach to improving the three C's is to make blocks larger to reduce the number
of compulsory misses, but, as we will see shortly, large blocks can increase other kinds of
misses.
The three C's give insight into the cause of misses, but this simple model has its limits; it
gives you insight into average behavior but may not explain an individual miss. For example,
changing cache size changes conflict misses as well as capacity misses, since a larger cache
spreads out references to more blocks. Thus, a miss might move from a capacity miss to a con-
lict miss as cache size changes. Note that the three C's also ignore replacement policy, since it
is difficult to model and since, in general, it is less significant. In specific circumstances the re-
placement policy can actually lead to anomalous behavior, such as poorer miss rates for larger
associativity, which contradicts the three C's model. (Some have proposed using an address
trace to determine optimal placement in memory to avoid placement misses from the three C's
model; we've not followed that advice here.)
Alas, many of the techniques that reduce miss rates also increase hit time or miss penalty.
The desirability of reducing miss rates using the three optimizations must be balanced against
the goal of making the whole system fast. This first example shows the importance of a bal-
anced perspective.
First Optimization: Larger Block Size To Reduce Miss Rate
The simplest way to reduce miss rate is to increase the block size. Figure B.10 shows the trade-
of of block size versus miss rate for a set of programs and cache sizes. Larger block sizes will
reduce also compulsory misses. This reduction occurs because the principle of locality has two
components: temporal locality and spatial locality. Larger blocks take advantage of spatial loc-
ality.