Cluster Management - Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset - page 246

Given a few simple SQL-like statements, you can build a highly functional dashboard using the Cloudera cluster
manager. Figure 8-26 shows the resource usage for the cluster server hc2nn, built using that simple SQL statement.
This type of dashboard generation function certainly proves useful for your cluster monitoring and management.
The Cloudera cluster manager and CDH 5 stack offer a great deal more functionality than can be covered in this
single chapter, but it will give you a sense of the services that the program can provide. Indeed, while developing
this topic, Cloudera has been my cluster manager for CDH5. For example, Cloudera enabled me to change the
configuration of my cluster when tuning performance. The early-warning orange icons at the top of Figure 8-18
helped me investigate problems as they arose, and so avoid future entanglements. Also, the software parcels were
especially useful when upgrading existing software and installing extra software. As I mentioned earlier, the costs in
terms of licensing are handily offset by the savings in problem solving.
Apache Bigtop
Although not a cluster manager itself, Apache Bigtop aims to simplify installation and integration in its own way.
Specifically, Bigtop is an attempt by the Apache Software Foundation to provide integration and smoke testing of the
Apache Hadoop tool kit in order to provide an integrated Hadoop tool stack. Through Bigtop, Apache selects multiple
Hadoop tools, each with its own release version, and uses a set of automated smoke tests to ensure that the set of
applications works together as a stack. The result is a well-tested, high-quality, stack-based Hadoop product set.
Cloudera has recognized the value that the Apache Bigtop project is adding by basing its CDH releases on the
Bigtop releases. Although Cloudera does its own testing for its CDH releases, that testing is based on a pre-tested
Bigtop Hadoop stack product.
In this section I show how to source the Bigtop software and install it. Given that the smoke tests are part of
the Apache Bigtop build, there isn't much to see. But I build Bigtop and extract the results of one smoke test as an
example.
Installing Bigtop
To install Bigtop, I obtain the Linux yum repository file so that Bigtop software can be sourced, I run the following
command on each machine in the cluster:
wget -O /etc/yum.repos.d/bigtop.repo http://archive.apache.org/dist/bigtop/bigtop-0.6.0/repos/
centos6/bigtop.repo
100%[============================================>] 172 --.-K/s in 0s
2014-08-27 18:30:40 (16.7 MB/s) - “/etc/yum.repos.d/bigtop.repo” saved [172/172]
As for installations of the Ambari and Cloudera cluster managers, I make sure the server has CentOS 6 freshly
installed. Next, I use the Linux yum command to install the components necessary for Bigtop on each machine:
yum install -y git cmake git-core git-svn subversion checkinstall build-essential dh-make debhelper
ant ant-optional autoconf automake liblzo2-dev libzip-dev sharutils libfuse-dev reprepro libtool
libssl-dev asciidoc xmlto ssh curl gcc gcc-c++ make fuse protobuf-compiler autoconf automake libtool
shareutils asciidoc xmlto lzo-devel zlib-devel fuse-devel openssl-devel python-devel libxml2-devel
libxslt-devel cyrus-sasl-devel sqlite-devel mysql-devel openldap-devel rpm-build create-repo redhat-
rpm-config wget