Hardware Reference
In-Depth Information
One of the most effective ways to improve both bandwidth and latency is to
use multiple caches. A basic technique that works very effectively is to introduce a
separate cache for instructions and data. There are several benefits from having
separate caches for instructions and data, often called a
split cache
. First, memory
operations can be initiated independently in each cache, effectively doubling the
bandwidth of the memory system. This is why it makes sense to provide two sepa-
rate memory ports, as we did in the Mic-1: each port has its own cache. Note that
each cache has independent access to the main memory.
Today, many memory systems are more complicated than this, and an addi-
tional cache, called a
level 2 cache
, may reside between the instruction and data
caches and main memory. In fact, as more sophisticated memory systems are re-
quired, there may be three or more levels of cache. In Fig. 4-37 we see a system
with three levels of cache. The CPU chip itself contains a small instruction cache
and a small data cache, typically 16 KB to 64 KB. Then there is the level 2 cache,
which is not on the CPU chip but may be included in the CPU package, next to the
CPU chip and connected to it by a high-speed path. This cache is generally uni-
fied, containing a mix of data and instructions. A typical size for the L2 cache is
512 KB to 1 MB. The third-level cache is on the processor board and consists of a
few megabytes of SRAM, which is much faster than the main DRAM memory.
Caches are generally inclusive, with the full contents of the level 1 cache being in
the level 2 cache and the full contents of the level 2 cache being in the level 3
cache.
CPU
package
CPU chip
Unified
L2
cache
Unified
L3 cache
L1-I L1-D
Main
memory
(DRAM)
Processor
board
Keyboard
controller
Graphics
controller
Disk
controller
Split L1 instruction and data caches
Board-level cache (SRAM)
Figure 4-37.
A system with three levels of cache.
To achieve their goal, caches depend on two kinds of address locality.
Spatial
locality
is the observation that memory locations with addresses numerically simi-
lar to a recently accessed memory location are likely to be accessed in the near
