Hardware Reference
In-Depth Information
Using cache memory reduces a traditional system bottleneck because system RAM is almost always
much slower than the CPU; the performance difference between memory and CPU speed has become
especially large in recent systems. Using cache memory prevents the processor from having to wait
for code and data from much slower main memory, thus improving performance. Without the L1
cache, a processor would frequently be forced to wait until system memory caught up.
Cache is even more important in modern processors because it is often the only memory in the entire
system that can truly keep up with the chip. Most modern processors are clock multiplied, which
means they are running at a speed that is really a multiple of the motherboard into which they are
plugged. The only types of memory matching the full speed of the processor are the L1, L2, and L3
caches built into the processor core.
See the
Chapter 6
section, “
Memory Module Speed
,”
p.
357
.
If the data that the processor wants is already in L1 cache, the CPU does not have to wait. If the data
is not in the cache, the CPU must fetch it from the Level 2 or Level 3 cache or (in less sophisticated
system designs) from the system bus—meaning main memory directly.
How Cache Works
To learn how the L1 cache works, consider the following analogy.
This story involves a person (in this case, you) eating food to act as the processor requesting and
operating on data from memory. The kitchen where the food is prepared is the main system memory
(typically double data rate [DDR], DDR2, or DDR3 dual inline memory module [DIMMs]). The
cache controller is the waiter, and the L1 cache is the table where you are seated.
Okay, here's the story. Say you start to eat at a particular restaurant every day at the same time. You
come in, sit down, and order a hot dog. To keep this story proportionately accurate, let's say you
normally eat at the rate of one bite (byte? <grin>) every four seconds (233MHz = about 4ns cycling).
It also takes 60 seconds for the kitchen to produce any given item that you order (60ns main memory).
So, when you arrive, you sit down, order a hot dog, and you have to wait for 60 seconds for the food
to be produced before you can begin eating. After the waiter brings the food, you start eating at your
normal rate. Pretty quickly you finish the hot dog, so you call the waiter over and order a hamburger.
Again, you wait 60 seconds while the hamburger is being produced. When it arrives, you again begin
eating at full speed. After you finish the hamburger, you order a plate of fries. Again you wait, and
after the fries are delivered 60 seconds later, you eat them at full speed. Finally, you decide to finish
the meal and order cheesecake for dessert. After another 60-second wait, you can eat cheesecake at
full speed. Your overall eating experience consists of a lot of waiting, followed by short bursts of
actual eating at full speed.
After coming into the restaurant for two consecutive nights at exactly 6 p.m. and ordering the same
items in the same order each time, on the third night the waiter begins to think, “I know this guy is
going to be here at 6 p.m., order a hot dog, a hamburger, fries, and then cheesecake. Why don't I have
these items prepared in advance and surprise him? Maybe I'll get a big tip.” So you enter the
restaurant and order a hot dog, and the waiter immediately puts it on your plate, with no waiting! You
then proceed to finish the hot dog and right as you are about to request the hamburger, the waiter
deposits one on your plate. The rest of the meal continues in the same fashion, and you eat the entire
meal, taking a bite every four seconds, and you never have to wait for the kitchen to prepare the food.
Your overall eating experience this time consists of all eating, with no waiting for the food to be
prepared, due primarily to the intelligence and thoughtfulness of your waiter.
