Design Patterns for Scaling - The Practice of Cloud System Administration - page 113

Another benefit of the queuing model is that it is easier to implement a priority scheme.
High-priority requests can go to the head of the queue. A plethora of queueing algorithms
may be available depending on the type of priority scheme you want to implement. In fair
queueing , the algorithm prevents a low-priority item from being “starved” by a flood of
high-priority items. Otheralgorithms dynamically shiftpriorities sothat burstyorintermit-
tent traffic does not overload the system or starve other priorities from being processed.
5.7.2 Variations
Variationsofthequeueingmodelcanoptimizeperformance.Itiscommontohavetheabil-
ity to shrink or grow the number of threads. This may be automatic based on demand, or
it may be manually controlled. Another variation is for threads to kill and re-create them-
selves periodically so that they remain “fresh.” This mitigates memory leaks and other
problems, but in doing so hides them and makes them more difficult to find.
Finally, it is common practice to use processes instead of threads. Process creation can
be expensive, but the fixed population of worker processes means that you pay that over-
head once and then get the benefit of using processes. Processes can do things that threads
cannot,becausetheyhavetheirownaddressspace,memory,andopenfiletables.Processes
are self-isolating, in that a process that is corrupted cannot hurt other processes, whereas
one ill-behaved thread can adversely affect other threads.
An example of queueing implemented with processes is the Prefork processing module
for the Apache web server. On startup, Apache forks off a certain number of subprocesses.
Requestsaredistributedtosubprocessesbyamasterprocess.Requestsareprocessedfaster
because the subprocess already exists, which hides the long process creation time. Pro-
cesses are configured to die and be refreshed to every n requests so that memory leaks are
averted. The number of subprocesses used can be adjusted dynamically.
5.8 Content Delivery Networks
A content delivery network (CDN) is a web-acceleration service that delivers content
(webpages,images,video)moreefficientlyonbehalfofyourservice.CDNscachecontent
on servers all over the world. Requests for content are serviced from the cache nearest the
user. Geolocation techniques are used to identify the network location of the requesting
web browser.
CDNs do not copy all content to all caches. Instead, they notice usage trends and de-
termine where to cache certain content. For example, seeing a surge of use from Germany
for a particular image, the CDN might copy all images for that customer to its servers in
Germany. These images may displace cached images that have not been accessed as re-
cently.