Thread-Level Parallelism - Computer Architecture: A Quantitative Approach - page 381

FIGURE 5.20 A directory is added to each node to implement cache coherence in a
distributed-memory multiprocessor . In this case, a node is shown as a single multicore
chip, and the directory information for the associated memory may reside either on or off the
multicore. Each directory is responsible for tracking the caches that share the memory ad-
dresses of the portion of memory in the node. The coherence mechanism would handle both
the maintenance of the directory information and any coherence actions needed within the
multicore node.
The simplest directory implementations associate an entry in the directory with each
memory block. In such implementations, the amount of information is proportional to the
product of the number of memory blocks (where each block is the same size as the L2 or L3
cache block) times the number of nodes, where a node is a single multicore processor or a
small collection of processors that implements coherence internally. This overhead is not a
problem for multiprocessors with less than a few hundred processors (each of which might
be a multicore) because the directory overhead with a reasonable block size will be tolerable.
For larger multiprocessors, we need methods to allow the directory structure to be eiciently
scaled, but only supercomputer-sized systems need to worry about this.
Directory-Based Cache Coherence Protocols: The Basics
Just as with a snooping protocol, there are two primary operations that a directory protocol
must implement: handling a read miss and handling a write to a shared, clean cache block.
(Handling a write miss to a block that is currently shared is a simple combination of these
two.) To implement these operations, a directory must track the state of each cache block. In a
simple protocol, these states could be the following:
■ Shared —One or more nodes have the block cached, and the value in memory is up to date
(as well as in all the caches).
■ Uncached —No node has a copy of the cache block.