Database Reference
In-Depth Information
A .NET map-reduce program comprises a number of parts (which are described in Table
5-1
):
•
Job definition
•
Mapper, Reducer, and Combiner classes
•
Input data
•
Job executor
Table 5-1.
The function of .NET MapReduce components
Component
Function
Job definition
This class has the declarations for Mapper, Reducer, and Combiner
types as well as the job configuration settings.
Map, Reduce, and Combine
These are the actual classes you use to implement your processing logic.
Input data
The data for the MapReduce job to process.
The entry point of your program—for example, the
Main()
method—which invokes the HadoopJobExecutor API.
Job executor
In the following section, you will create a MapReduce program that calculates the square root of all the integer
values provided as input and writes the output data to the file system.
In your
HadoopClient
solution, add three classes—
SquareRootMapper
,
SquareRootReducer
, and
SquareRootJob
—as shown in Figure
5-2
.
Figure 5-2.
Mapper, Reducer, and Job classes
You need to inherit your mapper class from the .NET Framework base class,
MapperBase
, and override its
Map()
method. Listing 5-2 shows the code for the mapper class.
