Designing in a Distributed World - The Practice of Cloud System Administration - page 9

Any fan in situation is at risk of having congestion problems. Often small queries may
resultinlargeresponses.Thereforeasmallamountofbandwidthisusedtofanoutbutthere
may not be enough bandwidth to sustain the fan in. This may result in congested network
links and overloaded servers. It is easy to engineer the system to have the right amount of
network and server capacity if the sizes of the queries and replies are consistent, or if there
isanoccasionallargereply.Thedifficultsituationisengineeringthesystemwhenthereare
sudden, unpredictable bursts of large replies. Some network equipment is engineered spe-
cifically to deal with this situation by dynamically provisioning more buffer space to such
bursts. Likewise, the backends can rate-limit themselves to avoid creating the situation in
the first place. Lastly, the frontends can manage the congestion themselves by controlling
the new queries they send out, by notifying the backends to slow down, or by implement-
ing emergency measures to handle the flood better. The last option is discussed in Chapter
5 .
1.3.3 Server Tree
The other fundamental composition pattern is the server tree . As Figure 1.3 illustrates, in
this scheme a number of servers work cooperatively with one as the root of the tree, parent
servers below it, and leaf servers at the bottom of the tree. (In computer science, trees are
drawn upside-down.) Typically this pattern is used to access a large dataset or corpus . The
corpus is larger than any one machine can hold; thus each leaf stores one fraction or shard
of the whole.