Database Reference
In-Depth Information
Listing 6-2.
Running the word-count MapReduce job from the Hadoop command line
hadoop jar hadoop-examples.jar wordcount /example/data/gutenberg/davinci.txt
/example/data/commandlineoutput
This launches the MapReduce job on the input file, and you should see an output similar to Listing 6-3.
Listing 6-3.
MapReduce command-line output
13/12/09 22:33:42 INFO input.FileInputFormat: Total input paths to process : 1
13/12/09 22:33:42 WARN snappy.LoadSnappy: Snappy native library is available
13/12/09 22:33:42 INFO util.NativeCodeLoader: Loaded the native-hadoop library
13/12/09 22:33:42 INFO snappy.LoadSnappy: Snappy native library loaded
13/12/09 22:33:43 INFO mapred.JobClient: Running job: job_201311240635_0196
13/12/09 22:33:44 INFO mapred.JobClient: map 0% reduce 0%
13/12/09 22:33:55 INFO mapred.JobClient: map 100% reduce 0%
13/12/09 22:34:03 INFO mapred.JobClient: map 100% reduce 33%
13/12/09 22:34:05 INFO mapred.JobClient: map 100% reduce 100%
13/12/09 22:34:07 INFO mapred.JobClient: Job complete: job_201311240635_0196
13/12/09 22:34:07 INFO mapred.JobClient: Counters: 29
13/12/09 22:34:07 INFO mapred.JobClient: Job Counters
13/12/09 22:34:07 INFO mapred.JobClient: Launched reduce tasks=1
13/12/09 22:34:07 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=8968
13/12/09 22:34:07 INFO mapred.JobClient: Total time spent by all reduces waiting
after reserving slots (ms)=0
13/12/09 22:34:07 INFO mapred.JobClient: Total time spent by all maps waiting after
reserving slots (ms)=0
13/12/09 22:34:07 INFO mapred.JobClient: Launched map tasks=1
13/12/09 22:34:07 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=10562
13/12/09 22:34:07 INFO mapred.JobClient: File Output Format Counters
13/12/09 22:34:07 INFO mapred.JobClient: Bytes Written=337623
13/12/09 22:34:07 INFO mapred.JobClient: FileSystemCounters
13/12/09 22:34:07 INFO mapred.JobClient: WASB_BYTES_READ=1395666
13/12/09 22:34:07 INFO mapred.JobClient: FILE_BYTES_READ=466915
13/12/09 22:34:07 INFO mapred.JobClient: HDFS_BYTES_READ=161
13/12/09 22:34:07 INFO mapred.JobClient: FILE_BYTES_WRITTEN=1057448
13/12/09 22:34:07 INFO mapred.JobClient: WASB_BYTES_WRITTEN=337623
13/12/09 22:34:07 INFO mapred.JobClient: File Input Format Counters
13/12/09 22:34:07 INFO mapred.JobClient: Bytes Read=1395667
13/12/09 22:34:07 INFO mapred.JobClient: Map-Reduce Framework
13/12/09 22:34:07 INFO mapred.JobClient: Map output materialized bytes=466761
13/12/09 22:34:07 INFO mapred.JobClient: Map input records=32118
13/12/09 22:34:07 INFO mapred.JobClient: Reduce shuffle bytes=466761
13/12/09 22:34:07 INFO mapred.JobClient: Spilled Records=65912
13/12/09 22:34:07 INFO mapred.JobClient: Map output bytes=2387798
13/12/09 22:34:07 INFO mapred.JobClient: Total committed heap usage
(bytes)=1029046272
13/12/09 22:34:07 INFO mapred.JobClient: CPU time spent (ms)=7420
13/12/09 22:34:07 INFO mapred.JobClient: Combine input records=251357
13/12/09 22:34:07 INFO mapred.JobClient: SPLIT_RAW_BYTES=161
13/12/09 22:34:07 INFO mapred.JobClient: Reduce input records=32956
13/12/09 22:34:07 INFO mapred.JobClient: Reduce input groups=32956
