Hardware Reference
In-Depth Information
This analogy describes the function of the L1 cache in the processor. The L1 cache itself is a table
that can contain one or more plates of food. Without a waiter, the space on the table is a simple food
buffer. When it's stocked, you can eat until the buffer is empty, but nobody seems to be intelligently
refilling it. The waiter is the cache controller who takes action and adds the intelligence to decide
which dishes are to be placed on the table in advance of your needing them. Like the real cache
controller, he uses his skills to literally guess which food you will require next, and if he guesses
correctly, you never have to wait.
Let's now say on the fourth night you arrive exactly on time and start with the usual hot dog. The
waiter, by now really feeling confident, has the hot dog already prepared when you arrive, so there is
no waiting.
Just as you finish the hot dog, and right as he is placing a hamburger on your plate, you say “Gee, I'd
really like a bratwurst now; I didn't actually order this hamburger.” The waiter guessed wrong, and
the consequence is that this time you have to wait the full 60 seconds as the kitchen prepares your
brat. This is known as a
cache miss
, in which the cache controller did not correctly fill the cache with
the data the processor actually needed next. The result is waiting, or in the case of a sample 233MHz
Pentium system, the system essentially throttles back to 16MHz (RAM speed) whenever a cache miss
occurs.
According to Intel, the L1 cache in most of its processors has approximately a 90% hit ratio. (Some
processors, such as the Pentium 4, are slightly higher.) This means that the cache has the correct data
90% of the time, and consequently the processor runs at full speed (233MHz in this example) 90% of
the time. However, 10% of the time the cache controller guesses incorrectly, and the data has to be
retrieved out of the significantly slower main memory, meaning the processor has to wait. This
essentially throttles the system back to RAM speed, which in this example was 60ns or 16MHz.
In this analogy, the processor was 14 times faster than the main memory. Memory speeds have
increased from 16MHz (60ns) to 333MHz (3.0ns) or faster in the latest systems, but processor speeds
have also risen to 3GHz and beyond. So even in the latest systems, memory is still 7.5 or more times
slower
than the processor. Cache is what makes up the difference.
The main feature of L1 cache is that it has always been integrated into the processor core, where it
runs at the same speed as the core. This, combined with the hit ratio of 90% or greater, makes L1
cache important for system performance.
Level 2 Cache
To mitigate the dramatic slowdown every time an L1 cache miss occurs, a secondary (L2) cache is
employed.
Using the restaurant analogy I used to explain L1 cache in the previous section, I'll equate the L2
cache to a cart of additional food items placed strategically in the restaurant such that the waiter can
retrieve food from the cart in only 15 seconds (versus 60 seconds from the kitchen). In an actual
Pentium class (Socket 7) system, the L2 cache is mounted on the motherboard, which means it runs at
motherboard speed (66MHz, or 15ns in this example). Now, if you ask for an item the waiter did not
bring in advance to your table, instead of making the long trek back to the kitchen to retrieve the food
and bring it back to you 60 seconds later, he can first check the cart where he has placed additional
items. If the requested item is there, he will return with it in only 15 seconds. The net effect in the real
system is that instead of slowing down from 233MHz to 16MHz waiting for the data to come from the
60ns main memory, the system can instead retrieve the data from the 15ns (66MHz) L2 cache. The
