Database Reference
In-Depth Information
The
run
method also sets the input and output paths for the job:
154 FileInputFormat.setInputPaths(conf, new Path(other_args.get(0)));
155 FileOutputFormat.setOutputPath(conf, new Path(other_args.get(1)));
Finally, line 157 runs the job:
157 JobClient.runJob(conf);
Running the Example 2 Code
To run this second Java example, you copy its file to the WordCount.java file so that the file name matches the Java class.
[hadoop@hc1nn wordcount]$ cp wc-ex2.java WordCount.java
Then, you remove the contents of the wc_classes directory and re-create it to receive the Java build output. Use
the Linux
rm
command for the Remove with
r
for “recursive”and
f
for “force switches.” Use the Linux
mkdir
command
to re-create the directory:
[hadoop@hc1nn wordcount]$ rm -rf wc_classes
[hadoop@hc1nn wordcount]$ mkdir wc_classes
You build the WordCount java file by specifying an output directory called wc_classes:
[hadoop@hc1nn wordcount]$ javac -classpath $HADOOP_PREFIX/hadoop-core-1.2.1.jar -d wc_classes
WordCount.java
Then, you list the contents of the wc_classes directory recursively to ensure that the org.myorg directory structure
exists and contains the newly compiled classes:
[hadoop@hc1nn wordcount]$ ls -R wc_classes
wc_classes:
org
wc_classes/org:
myorg
wc_classes/org/myorg:
WordCount.class WordCount$Map.class WordCount$Map$Counters.class WordCount$Reduce.class
You build these classes into a jar library called wordcount1.jar, so that the resulting jar file can be used for a
Hadoop Map Reduce job run. Use the Linux
jar
command for this (which operates in a similar manner to
tar
) by
using the options
C
for “create,”
v
for “verbose,” and
f
to specify the file to create:
[hadoop@hc1nn wordcount]$ jar -cvf ./wordcount1.jar -C wc_classes.
added manifest
adding: org/(in = 0) (out= 0)(stored 0%)
adding: org/myorg/(in = 0) (out= 0)(stored 0%)
adding: org/myorg/WordCount.class(in = 2671) (out= 1289)(deflated 51%)
adding: org/myorg/WordCount$Reduce.class(in = 1611) (out= 648)(deflated 59%)
adding: org/myorg/WordCount$Map$Counters.class(in = 983) (out= 504)(deflated 48%)
adding: org/myorg/WordCount$Map.class(in = 4661) (out= 2217)(deflated 52%)
Search WWH ::
Custom Search