Database Reference
In-Depth Information
The unique
Block Pool ID
number signifies that the namenodes manage their own block
pools and do not interfere with each other's operations. The cluster is now configured with
a federated HDFS. Let's now try to test the two namenodes using the following steps:
1. Create a folder on the namespace managed by the namenode on
node1.hcluster
using the following command:
$ sudo -u hdfs hdfs dfs -mkdir
hdfs://node1.hcluster:8020/node1_data
2. Create a folder on the namespace managed by the namenode on
node2.hcluster
using the following command:
$ sudo -u hdfs hdfs dfs -mkdir
hdfs://node2.hcluster:8020/node2_data
3. Next, let's list the contents of the two namespaces. To list the contents of the
namespace hosted on
node1.hcluster
and
node2.hcluster
, use the
commands from the following screenshot:
Configuring ViewFS for a federated HDFS
As you can see for the commands in the preceding screenshot, each namenode was re-
ferred to by its name. This is not feasible as each client would need to know the name of
each namenode and the files it manages. This is where
ViewFS
comes in. It helps define
paths to the different namenodes by using a mount table configuration. The mount table is
configured in the
core-site.xml
file.
Edit the
core-site.xml
file by performing the following updates:
1. Update the
fs.defaultFS
property to
viewfs:///
as shown in the follow-
ing code: