Database Reference
In-Depth Information
ZooKeeper in Production
In production, you should run ZooKeeper in replicated mode. Here, we will cover some of
the considerations for running an ensemble of ZooKeeper servers. However, this section is
not exhaustive, so you should consult the
ZooKeeper Administrator's Guide
for detailed,
up-to-date instructions, including supported platforms, recommended hardware, mainten-
ance procedures, dynamic reconfiguration (to change the servers in a running ensemble),
and configuration properties.
Resilience and Performance
ZooKeeper machines should be located to minimize the impact of machine and network
failure. In practice, this means that servers should be spread across racks, power supplies,
and switches, so that the failure of any one of these does not cause the ensemble to lose a
majority of its servers.
For applications that require low-latency service (on the order of a few milliseconds), it is
important to run all the servers in an ensemble in a single data center. Some use cases don't
require low-latency responses, however, which makes it feasible to spread servers across
data centers (at least two per data center) for extra resilience. Example applications in this
category are leader election and distributed coarse-grained locking, both of which have rel-
atively infrequent state changes, so the overhead of a few tens of milliseconds incurred by
inter-data-center messages is not significant relative to the overall functioning of the ser-
vice.
NOTE
ZooKeeper has the concept of an
observer node
, which is like a non-voting follower. Because they do not
participate in the vote for consensus during write requests, observers allow a ZooKeeper cluster to im-
prove read performance without hurting write performance.
[
151
]
Observers can be used to good advantage
to allow a ZooKeeper cluster to span data centers without impacting latency as much as regular voting fol-
lowers. This is achieved by placing the voting members in one data center and observers in the other.
ZooKeeper is a highly available system, and it is critical that it can perform its functions in
a timely manner. Therefore, ZooKeeper should run on machines that are dedicated to
ZooKeeper alone. Having other applications contend for resources can cause ZooKeeper's
performance to degrade significantly.
Configure ZooKeeper to keep its transaction log on a different disk drive from its snap-
shots. By default, both go in the directory specified by the
dataDir
property, but if you
specify a location for
dataLogDir
, the transaction log will be written there. By having
