Design Patterns for Scaling - The Practice of Cloud System Administration - page 104

5.4.1 Cache Effectiveness
The effectiveness of a cache is measured by the cache hit ratio , sometimes called a cache
hit rate . It is the ratio of the number of cache hits over the total number of lookups. For
example,if500lookupsareperformedand100wereservicedfromthecache,thecachehit
ratio would be 1/5 or 20 percent.
Performance
Acacheisanetbenefitinperformanceifthetimesavedduringcachehitsexceedsthetime
lostfromtheadditionaloverhead.Wecanestimatethisusingweightedaverages.Ifthetyp-
ical time for a regular lookup is L , a cache hit is H , a cache miss is M , and the cache hit
ratio is R , then using the cache is more effective if H × R + M × (1 − R ) < L .
When doing engineering estimates we can simplify the formula if cache lookups and
updates are extremely fast or nearly zero. In that case we can assume the performance be-
nefit will be a function of the cache hit ratio. For example, suppose a typical lookup took 6
secondsandwepredictacachehitrateof33percent.Weknowthatinaperfectworldwith
an instant, zero-overhead cache, the average lookup would fall by 33 percent to 4 seconds.
Thisgivesusabest-casescenario,whichisusefulforplanningpurposes.Realistically,per-
formance will improve slightly less than this ideal.
Cost-Effectiveness
Acacheiscost-effectiveonlyifthebenefitfromthecacheisgreaterthanthecostofimple-
mentingthecache.RecallthataccessingRAMisfasterthanaccessingdisk,butmuchmore
expensive. The performance and costs of the items in Figure 1.10 will inevitably change
over time. We recommend building your own chart based on the performance and costs in
your specific environment.
Suppose a system without caching required 20 replicas, but with caching required only
15. If each replica is a machine, this means the cache is more cost-effective if it costs less
than purchasing 5 machines.
Purchasepriceisnotalwaystheonlyconsideration.Ifbeingabletoprovideafasterser-
vice will improve sales by 20 percent, then cost should be weighed against that improve-
ment.