Review of Memory Hierarchy - Computer Architecture: A Quantitative Approach - page 564

Hence, the split caches in this example—which offer two memory ports
per clock cycle, thereby avoiding the structural hazard—have a beter average
memory access time than the single-ported unified cache despite having a
worse effective miss rate.
Average Memory Access Time And Processor Performance
An obvious question is whether average memory access time due to cache misses predicts pro-
cessor performance.
First, there are other reasons for stalls, such as contention due to I/O devices using memory.
Designers often assume that all memory stalls are due to cache misses, since the memory hier-
archy typically dominates other reasons for stalls. We use this simplifying assumption here,
but be sure to account for all memory stalls when calculating final performance.
Second, the answer also depends on the processor. If we have an in-order execution pro-
cessor (see Chapter 3 ), then the answer is basically yes. The processor stalls during misses, and
the memory stall time is strongly correlated to average memory access time. Let's make that
assumption for now, but we'll return to out-of-order processors in the next subsection.
As stated in the previous section, we can model CPU time as:
This formula raises the question of whether the clock cycles for a cache hit should be con-
sidered part of CPU execution clock cycles or part of memory stall clock cycles. Although
either convention is defensible, the most widely accepted is to include hit clock cycles in CPU
execution clock cycles.
We can now explore the impact of caches on performance.
Example
Let's use an in-order execution computer for the first example. Assume that the
cache miss penalty is 200 clock cycles, and all instructions normally take 1.0
clock cycles (ignoring memory stalls). Assume that the average miss rate is 2%,
there is an average of 1.5 memory references per instruction, and the average
number of cache misses per 1000 instructions is 30. What is the impact on per-
formance when behavior of the cache is included? Calculate the impact using
both misses per instruction and miss rate.
Answer