Database Reference
In-Depth Information
Chapter 3
Service Configuration and Coordination
The early development of high-performance computing was primarily
focused on scale-up architectures. These systems had multiple processors
with a shared memory bus. In fact, modern multicore server hardware is
architecturally quite similar to the supercomputer hardware of the 1980s
(although modern hardware usually foregoes the Cray's bench seating).
As system interlinks and network hardware improved, these systems began
to become more distributed, eventually evolving into the Network of
Workstations (NOW) computing environments used in most present-day
computing environments. However, these interlinks usually do not have the
bandwidth required to simulate the shared resource environments of a
scale-up architecture nor do they have the reliability of interlinks used in a
scale-up environment. Distributed systems also introduce new failure modes
that are not generally found in a scale-up environment.
This chapter discusses systems and techniques used to overcome these
problems in a distributed environment, in particular, the management of
shared state in distributed applications. Configuration management and
coordination is first discussed in general. The most popular system, used
by most of the other software in this topic, is ZooKeeper. This system was
developed by Yahoo! to help manage its Hadoop infrastructure and is
discussed extensively because it is used by most of the other distributed
software systems in this topic.
Motivation for Configuration and Coordination
Systems
Most distributed systems need to share some system metadata as well as
some state concerning the distributed system itself. The metadata is typically
some sort of configuration information. For example, it might need to store
the location ofvarious database shards in a distributed server. System state is
usually data used to coordinate the application. For example, a master-slave
system needs to keep track of the server currently acting as the master.
Furthermore, this happens in a relatively unreliable environment, so a server
