Hardware Reference
In-Depth Information
After coming into the restaurant for two consecutive nights at exactly 6 p.m. and ordering
the same items in the same order each time, on the third night the waiter begins to think,
“Iknowthisguyisgoingtobehereat6p.m.,orderahotdog,ahamburger,fries,andthen
cheesecake. Why don't I have these items prepared in advance and surprise him? Maybe
I'll get a big tip.” So you enter the restaurant and order a hot dog, and the waiter imme-
diately puts it on your plate, with no waiting! You then proceed to finish the hot dog and
rightasyouareabouttorequestthehamburger,thewaiterdepositsoneonyourplate.The
rest of the meal continues in the same fashion, and you eat the entire meal, taking a bite
every four seconds, and you never have to wait for the kitchen to prepare the food. Your
overall eating experience this time consists of all eating, with no waiting for the food to
be prepared, due primarily to the intelligence and thoughtfulness of your waiter.
This analogy describes the function of the L1 cache in the processor. The L1 cache itself
is a table that can contain one or more plates of food. Without a waiter, the space on the
table is a simple food buffer. When it's stocked, you can eat until the buffer is empty, but
nobody seems to be intelligently refilling it. The waiter is the cache controller who takes
action and adds the intelligence to decide which dishes are to be placed on the table in
advance of your needing them. Like the real cache controller, he uses his skills to liter-
allyguesswhichfoodyouwillrequirenext,andifheguessescorrectly,youneverhaveto
wait.
Let's now say on the fourth night you arrive exactly on time and start with the usual hot
dog. The waiter, by now really feeling confident, has the hot dog already prepared when
you arrive, so there is no waiting.
Justasyoufinishthehotdog,andrightasheisplacingahamburgeronyourplate,yousay
“Gee, I'd really like a bratwurst now; I didn't actually order this hamburger.” The waiter
guessed wrong, and the consequence is that this time you have to wait the full 60 seconds
as the kitchen prepares your brat. This is known as a
cache miss
, in which the cache con-
trollerdidnotcorrectlyfillthecachewiththedatatheprocessoractuallyneedednext.The
result is waiting, or in the case of a sample 233MHz Pentium system, the system essen-
tially throttles back to 16MHz (RAM speed) whenever a cache miss occurs.
According to Intel, the L1 cache in most of its processors has approximately a 90% hit
ratio. (Some processors, such as the Pentium 4, are slightly higher.) This means that the
cache has the correct data 90% of the time, and consequently the processor runs at full
speed (233MHz in this example) 90% of the time. However, 10% of the time the cache
controller guesses incorrectly, and the data has to be retrieved out of the significantly
slowermainmemory,meaningtheprocessorhastowait.Thisessentiallythrottlesthesys-
tem back to RAM speed, which in this example was 60ns or 16MHz.
