Designing in a Distributed World - The Practice of Cloud System Administration - page 7

Trouble with a Naive Least Loaded Algorithm
Without slow start, load balancers have been known to cause many problems. One
famous example is what happened to the CNN.com web site on the day of the
September 11, 2001, terrorist attacks. So many people tried to access CNN.com
that the backends became overloaded. One crashed, and then crashed again after it
came back up, because the naive least loaded algorithm sent all traffic to it. When
it was down, the other backends became overloaded and crashed. One at a time,
each backend would get overloaded, crash, and become overloaded from again re-
ceiving all the traffic and crash again.
As a result the service was essentially unavailable as the system administrators
rushed to figure out what was going on. In their defense, the web was new enough
that no one had experience with handling sudden traffic surges like the one en-
countered on September 11.
The solution CNN used was to halt all the backends and boot them at the same
time so they would all show zero load and receive equal amounts of traffic.
The CNN team later discovered that a few days prior, a software upgrade for
their load balancer hadarrived buthadnotyet been installed. The upgrade added a
slow start mechanism.
1.3.2 Server with Multiple Backends
The next composition pattern is a server with multiple backends. The server receives a re-
quest, sends queries to many backend servers, and composes the final reply by combining
those answers. This approach is typically used when the original query can easily be de-
constructed into a number of independent queries that can be combined to form the final
answer.
Figure 1.2a illustrates how a simple search engine processes a query with the help of
multiple backends. The frontend receives the request. It relays the query to many backend
servers. The spell checker replies with information so the search engine may suggest al-
ternatespellings.Thewebandimagesearchbackendsreplywithalistofwebsitesandim-
ages related to the query. The advertisement server replies with advertisements relevant to
the query. Once the replies are received, the frontend uses this information to construct the
HTML that makes up the search results page for the user, which is then sent as the reply.