Designing in a Distributed World - The Practice of Cloud System Administration - page 8

Figure 1.2: This service is composed of a server and many backends.
Figure 1.2b illustrates the same architecture with replicated, load-balanced, backends.
The same principle applies but the system is able to scale and survive failures better.
This kind of composition has many advantages. The backends do their work in parallel.
Thereplydoesnothavetowaitforonebackendprocesstocompletebeforethenextbegins.
The system is loosely coupled. One backend can fail and the page can still be constructed
by filling in some default information or by leaving that area blank.
This pattern also permits some rather sophisticated latency management. Suppose this
system is expected to return a result in 200 ms or less. If one of the backends is slow for
some reason, the frontend doesn't have to wait for it. If it takes 10 ms to compose and send
theresultingHTML,at190msthefrontendcangiveupontheslowbackendsandgenerate
the page with the information it has. The ability to manage a latency time budget like that
can be very powerful. For example, if the advertisement system is slow, search results can
be displayed without any ads.
To be clear, the terms “frontend” and “backend” are a matter of perspective. The fron-
tend sends requests to backends, which reply with a result. A server can be both a frontend
and a backend. In the previous example, the server is the backend to the web browser but a
frontend to the spell check server.
There are many variations on this pattern. Each backend can be replicated for increased
capacity or resiliency. Caching may be done at various levels.
The term fan out refers to the fact that one query results in many new queries, one to
each backend. The queries “fan out” to the individual backends and the replies fan in as
they are set up to the frontend and combined into the final result.