Exploring the HDInsight Name Node - Pro Microsoft HDInsight: Hadoop on Windows - page 93

13/12/09 22:34:07 INFO mapred.JobClient: Combine output records=32956
13/12/09 22:34:07 INFO mapred.JobClient: Physical memory (bytes) snapshot=493834240
13/12/09 22:34:07 INFO mapred.JobClient: Reduce output records=32956
13/12/09 22:34:07 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1430384640
13/12/09 22:34:07 INFO mapred.JobClient: Map output records=251357

■ The jobs you execute from the .neT and powerShell programs are broken down internally as similar commands
and executed as command-line jobs.

Note

Make sure that the output files are created in the commandlineoutput folder as provided in the MapReduce
command by issuing another ls command. This command lists the output file(s) created by the job as in Listing 6-4.

Listing 6-4. Verifying the output
c:\apps\dist\hadoop-1.2.0.1.3.1.0-06>hdfs fs -ls \example\data\commandlineoutput

Found 1 items -rw-r--r-- 1 hadoopusersupergroup 337623 2013-12-09
22:34 /example/data/commandlineoutput/part-r-00000

You can copy output to the local file system and inspect the results (occurrences for each word will be in
c:\output\part-r-00000 ) using the command in Listing 6-5.

Listing 6-5. Copying the MapReduce output from HDFS to local file system

hadoop dfs -copyToLocal /example/data/commandlineoutput c:\output

You can use Windows Explorer to view the output folder in your C:\Output directory as shown in Figure 6-8 .

Figure 6-8. The output folder in the local file system

As indicated before, because Windows does not understand shell scripts for Linux ( .sh files), all the command
scripts and executables are implemented through Windows command files ( .cmd files). You can use them directly
from the command prompt as you would do in Linux, thus providing a complete abstraction to end users on
Windows. For example, to start or stop your cluster, you can use the commands:

• stop-master.cmd

• stop-slave.cmd