Database Reference
In-Depth Information
The JournalNodes are daemons that run on JournalNode machines. It is advisable to use at
least three JournalNode daemons to make sure the edit logs are written to three different
locations allowing JournalNodes to tolerate the failure of a single machine. The system
can tolerate the failure of
(N-1)/2 JournalNodes
, where
N
is the number of JournalNodes
configured.
At any time, only the active namenode can write to the JournalNodes. This is handled by
the JournalNodes to avoid the updates of the shared edit logs by two namenodes, which
could cause data loss as well as incorrect results. The standby namenode will only have
read access to the JournalNodes to update its own namespace.
When configuring HA, the secondary namenode is not used. As you will recall, the sec-
ondary namenode is a checkpoint service that performs periodic checkpoints of the edit
logs in the primary namenode and updates the fsimage. The standby namenode in an HA
environment performs the checkpoints and does not need a secondary namenode.
The datanodes of the cluster update both the active as well as the standby namenode with
the location and heartbeat information, thus making it possible for a quick failover.
When the active namenode fails, the standby namenode assumes the responsibility of
writing to the JournalNodes and takes over the active role.
Configuring HDFS high availability by theQuorum-based storage
There are two types of failover configurations: manual failover, which involves manually
initiating the commands for failover and automatic failover, where there is no manual in-
tervention.
To configure HDFS HA, a new property,
NamenodeID
is used. The
NamenodeID
is
used to distinguish each namenode in the cluster.
Let's look at the properties defined in
hdfs-site.xml
configuration file to set up
HDFS HA:
•
dfs.nameservices
: Just as we did for HDFS Federation, we need to configure the
nameservices in use for the cluster. For this configuration, I am not using a feder-
ated HDFS. The following is a sample configuration entry for
dfs.nameservices
:
