Database Reference
In-Depth Information
Shared storage using NFS
In this approach, the active and standby namenodes need to have access to a common
shared storage such as
Network File System
(
NFS
). The active namenode logs a copy of
its own namespace modifications on the shared network storage. The standby namenode
reads these modifications and applies them to its own namespace and keeps its namespace
in sync with the active namenode. In the event of a failover, the standby namenode would
have to wait till all the operations from the shared edits log have been applied to its
namespace before transitioning into the active state. All datanodes are configured to send to
block information and heartbeats to both namenodes. A fencing method needs to be de-
ployed for this approach to make sure that the previously active namenode is shut down be-
fore the standby namenode becomes active.
The hardware for the namenode machines in this architecture should be equivalent and just
like the hardware setup of a non-HA namenode. The shared storage should be accessible by
both the namenodes and should be configured for redundancy to handle failures. Redund-
ancy should be configured for the disk, network, and power. The access and hardware of
the shared storage are very critical to this architecture and should be of a high quality with
multiple network access paths. Redundancy prevents the NFS (shared storage) from be-
coming the single point of failure.
Configuring HDFS high availability by shared storage using NFS
Almost all the configuration parameters for the
hdfs-site.xml
configuration file are
similar to the one we did for Quorum-based storage. However, we need to update the prop-
erty as shown in the following code to set up HDFS HA using NFS:
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>file:///mnt/shared_storage</value>
</property>
Here, the property
dfs.namenode.shared.edits.dir
points to a shared directory,
which has been locally mounted.
Once you are done deciding and configuring the desired method for HDFS HA (Quorum-
based or Shared Storage), you need to perform the following steps:
