Database Reference
In-Depth Information
Chapter 4. Management and
Monitoring
Building and keeping tabs on a big data architecture can be a daunting task. You've got a di-
verse set of software spread out across many machines that might have dramatically different
configurations. How can you tell if part of your system is failing, and how do you bring that
component back up after you've fixed the problem? How can the different parts of your sys-
tem communicate with on another so they can do big jobs with many moving parts?
Fortunately, the big data ecosystem provides a variety of tools to ease the burden of man-
aging and monitoring your architecture. We're going to address three primary categories of
these tools:
Node configuration management
These are tools like Puppet or Chef that can help you manage the configuration of your
systems. They do things like change operating system parameters and install software.
Resource tracking
While many individual components in your architecture may come with tools to monitor
the performance of that specific component, sometimes you need a single dashboard or
insight into something that isn't tied to a specific tool.
Coordination
Many tasks take advantage of a handful of different components of your big data system.
Tools like ZooKeeper can help you synchronize all those moving parts to accomplish a
single goal.
