Database Reference
In-Depth Information
[root@hc1nn sysconfig]# service iptables restart
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Flushing firewall rules: [ OK ]
iptables: Unloading modules: [ OK ]
iptables: Applying firewall rules: [ OK ]
You are now ready to start the YARN servers. First, start the Resource Manager on the master, then start the Node
managers on the data nodes. Finally, check the logs for errors. The sequence of commands you need for doing this is
as follows:
[root@hc1nn init.d]# service hadoop-yarn-resourcemanager start
Starting Hadoop resourcemanager: [ OK ]
starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-hc1nn.out
[root@hc1r1m1 init.d]# service hadoop-yarn-nodemanager start
Starting Hadoop nodemanager: [ OK ]
starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-hc1r1m1.out
Running Another Map Reduce Job Test
You have the Hadoop V2 HDFS and YARN servers running; all servers are up on the data nodes and the Node
manager. You have checked the logs and found no errors. So, you are ready to attempt a test of Map Reduce. Try
issuing the word-count job on the Poe data, as was done earlier for Hadoop V1 :
1.
Switch user to the Linux hadoop account:
[root@hc1nn ~]$ su - hadoop
2.
Next, create the working directory for the data on HDFS:
[hadoop@hc1nn ~]$ hdfs dfs -mkdir -p /user/hadoop/edgar/edgar
3.
Now, copy the same edgar test data to the working directory on HDFS:
[hadoop@hc1nn edgar]$ hdfs dfs -copyFromLocal *.txt /user/hadoop/edgar/edgar
4.
And finally, run the Map Reduce word-count job:
[hadoop@hc1nn ~]$ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount /
user/hadoop/edgar/edgar /user/hadoop/edgar-results
14/03/23 16:34:46 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
14/03/23 16:34:48 INFO util.ProcessTree: setsid exited with exit code 0
14/03/23 16:34:48 INFO mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxR
esourceCalculatorPlugin@18b4587
14/03/23 16:34:48 INFO mapred.MapTask: Processing split: hdfs://hc1nn/user/hadoop/edgar/edgar/10947-8.
txt:0+632294
14/03/23 16:34:48 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.
MapTask$MapOutputBuffer
14/03/23 16:34:48 INFO mapred.MapTask: io.sort.mb = 100
Search WWH ::
Custom Search