Database Reference
In-Depth Information
Now it's time to start the Hadoop servers. On the Name Node machine (hc1nn), run the following as root to start
the HDFS, YARN, and History servers:
service hadoop-hdfs-namenode start
service hadoop-mapreduce-historyserver start
service hadoop-yarn-resourcemanager start
service hadoop-yarn-proxyserver start
On the Data Node machines (hc1r1m1 to hc1r1m3), run the following as root to start the data node and Node
Manager:
service hadoop-hdfs-datanode start
service hadoop-yarn-nodemanager start
You can make sure that Hadoop is writing to HDFS by executing the Hadoop file system ls command on
the /directory:
[root@hc1nn conf]# su - hdfs
-bash-4.1$ hadoop fs -ls /
Also, ensure that the temporary directory exists on HDFS:
-bash-4.1$ hadoop fs -mkdir /tmp
-bash-4.1$
-bash-4.1$ hadoop fs -chmod -R 1777 /tmp
The permissions “1777” means that any file system user can write to the directory but cannot remove another
user's files.
You need to create the required directories under HDFS; these directories will be owned and used by YARN for
logging and history data:
[root@hc1nn hadoop-hdfs]# su - hdfs
-bash-4.1$
-bash-4.1$ hadoop fs -mkdir /user/history
-bash-4.1$ hadoop fs -chmod -R 1777 /user/history
-bash-4.1$ hadoop fs -chown yarn /user/history
-bash-4.1$ hadoop fs -mkdir /var/log/hadoop-yarn
-bash-4.1$ hadoop fs -chown yarn:mapred /var/log/hadoop-yarn
At this point, check which top-level directories exist on the Hadoop distributed file system (HDFS). Because
you have executed a long list ( ls -l ) and have used a recursive switch ( -R ), you can also see which subdirectories
exist. The following listing shows permissions (drwxrwxrw), plus details like ownership and group membership
(hdfs hadoop) :
-bash-4.1$ hadoop fs -ls -R /
drwxrwxrwt - hdfs hadoop 0 2014-03-23 14:58 /tmp
drwxr-xr-x - hdfs hadoop 0 2014-03-23 14:55 /user
drwxrwxrwt - yarn hadoop 0 2014-03-23 14:55 /user/history
drwxr-xr-x - hdfs hadoop 0 2014-03-23 14:56 /var
drwxr-xr-x - hdfs hadoop 0 2014-03-23 14:56 /var/log
drwxr-xr-x - yarn mapred 0 2014-03-23 14:56 /var/log/hadoop-yarn
 
Search WWH ::




Custom Search