Database Reference
In-Depth Information
Implementing HDFS High Availability
Setting up a cluster is just one of the responsibilities of a Hadoop administrator. Once the
cluster is up and running, the administrator needs to make sure the environment is stable
and should handle downtime efficiently. Hadoop, being a distributed system, is not only
prone to failures, but is expected to fail. The master nodes such as the namenode and job-
tracker are single points of failure. A
single point of failure
(
SPOF
) is a system in the
cluster, if it fails, it causes the whole cluster to be nonfunctional. Having a system to handle
these single point failures is a must. We will be exploring the techniques on how to handle
namenode failures by configuring
HDFS HA
(
high availability
).
The namenode stores all the location information of the files in a cluster and coordinates
access to the data. If the namenode goes down, the cluster is unusable until the namenode is
brought up. Maintenance windows to upgrade hardware or software on the namenode could
also cause downtime. The secondary namenode, as we have already discussed, is a check-
point service and does not support automatic failover for the namenode.
The time taken to bring back the namenode online depends on the type of failure (hardware
and software). The downtime could result in
Service Level Agreement
(
SLA
) slippage as
well as the productivity of the data team. To handle such issues and make the namenode
more available, namenode HA was built and integrated into Apache Hadoop 2.0.0.
CDH5 comes with HDFS HA built-in. HDFS HA is achieved by running two namenodes
for a cluster in an active/passive configuration. In an active/passive configuration, only one
namenode is active at a time. When the active namenode becomes unavailable, the passive
namenode assumes responsibility and makes itself as the primary active namenode. In this
configuration, the two namenodes run on two physically different machines.
The two namenodes in an active/passive configuration are called active namenode and
standby namenode respectively. Both the active as well as the standby namenode need to
be of similar hardware.
Using CDH5, high availability can be configured using the following two techniques:
• Quorum-based storage
• Shared storage using NFS
