Databases Reference
In-Depth Information
30965 Jps
[hadoop-user@backup]$ jps
2099 Jps
1679 SecondaryNameNode
[hadoop-user@hadoop1]$ jps
7101 TaskTracker
7617 Jps
6988 DataNode
You have a functioning cluster!
Switching between modes
A practice that I found useful when starting with Hadoop was to use symbolic links
to switch between Hadoop modes instead of constantly editing the XML files. To
do so, create a separate configuration folder for each of the modes and place the
appropriate version of the XML files in the corresponding folder. Below is an example
directory listing:
[hadoop@hadoop_master hadoop]$ ls -l
total 4884
drwxr-xr-x 2 hadoop-user hadoop 4096 Nov 26 17:36 bin
-rw-rw-r-- 1 hadoop-user hadoop 57430 Nov 13 19:09 build.xml
drwxr-xr-x 4 hadoop-user hadoop 4096 Nov 13 19:14 c++
-rw-rw-r-- 1 hadoop-user hadoop 287046 Nov 13 19:09 CHANGES.txt
lrwxrwxrwx 1 hadoop-user hadoop 12 Jan 5 16:06 conf -> conf.cluster
drwxr-xr-x 2 hadoop-user hadoop 4096 Jan 8 17:05 conf.cluster
drwxr-xr-x 2 hadoop-user hadoop 4096 Jan 2 15:07 conf.pseudo
drwxr-xr-x 2 hadoop-user hadoop 4096 Dec 1 10:10 conf.standalone
drwxr-xr-x 12 hadoop-user hadoop 4096 Nov 13 19:09 contrib
drwxrwxr-x 5 hadoop-user hadoop 4096 Jan 2 09:28 datastore
drwxr-xr-x 6 hadoop-user hadoop 4096 Nov 26 17:36 docs
...
You can then switch between configurations by using the Linux
ln
command (e.g.,
ln -s conf.cluster conf
). This practice is also useful to temporarily pull a node
out of the cluster to debug a MapReduce program in pseudo-distributed mode, but
be sure that the modes have different file locations for HDFS and stop all daemons
on the node before changing configurations.
Now that we've gone through all the settings to successfully get a Hadoop cluster up
and running, we'll introduce the Web UI for basic monitoring of the cluster's state.
