Database Reference
In-Depth Information
•
dfs.namenode.name.dir
: This property defines the location of the fsimage
file on the local filesystem. If the value is specified as a comma-separated list of
locations, the fsimage file will be replicated on all of the locations for redund-
ancy.
•
dfs.namenode.edits.dir
: This property defines the location of the edits
log file. Again, if the value is specified as a comma-separated list of locations, the
edits log file will be replicated on all of the locations for redundancy.
•
dfs.namenode.checkpoint.dir
: This property defines the location of the
temporary filesystem images to be used by the secondary namenode. If the value
is specified as a comma-separated list of locations, the temporary filesystem im-
age file will be replicated on all of the locations for redundancy.
•
dfs.namenode.checkpoint.edits.dir
: This property defines the loca-
tion of the temporary edits log file to be used by the secondary namenode. If the
value is specified as a comma-separated list of locations, the temporary edits log
file will be replicated on all of the locations for redundancy.
These properties when configured are suffixed by the
NameserviceID
. For example, as-
suming that we have two nameservices:
ns1
and
ns2
, there would be two entries in the
hdfs-site.xml
file; one for
dfs.namenode.rpc-address.ns1
and another for
dfs.namenode.rpc-address.ns2
. Let's look at a practical configuration so you
understand these properties better.
As shown in the diagram earlier,
node1.hcluster
and
node2.hcluster
are the
nodes that will host the namenodes NN1 and NN2 respectively.
The following are the steps to configure HDFS Federation for a new cluster with the two
nodes as namenodes (
node1.hcluster
and
node2.hcluster
):
1. Update the
hdfs-site.xml
file to add the following properties:
<property>
<name>dfs.nameservices</name>
<value>ns1,ns2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1</name>
<value>node1.hcluster:8020</value>
</property>
<property>
