MapReduce - Hadoop: The Definitive Guide - page 30

perature is wrapped in an IntWritable . We write an output record only if the temper-
ature is present and the quality code indicates the temperature reading is OK.
The reduce function is similarly defined using a Reducer , as illustrated in Example 2-4 .
Example 2-4. Reducer for the maximum temperature example
import java.io.IOException ;
import org.apache.hadoop.io.IntWritable ;
import org.apache.hadoop.io.Text ;
import org.apache.hadoop.mapreduce.Reducer ;
public class MaxTemperatureReducer
extends Reducer < Text , IntWritable , Text , IntWritable > {
@Override
public void reduce ( Text key , Iterable < IntWritable > values , Context
context )
throws IOException , InterruptedException {
int maxValue = Integer . MIN_VALUE ;
for ( IntWritable value : values ) {
maxValue = Math . max ( maxValue , value . get ());
}
context . write ( key , new IntWritable ( maxValue ));
}
}
Again, four formal type parameters are used to specify the input and output types, this
time for the reduce function. The input types of the reduce function must match the output
types of the map function: Text and IntWritable . And in this case, the output types
of the reduce function are Text and IntWritable , for a year and its maximum tem-
perature, which we find by iterating through the temperatures and comparing each with a
record of the highest found so far.
The third piece of code runs the MapReduce job (see Example 2-5 ) .
Example 2-5. Application to find the maximum temperature in the weather dataset
import org.apache.hadoop.fs.Path ;
import org.apache.hadoop.io.IntWritable ;
import org.apache.hadoop.io.Text ;
import org.apache.hadoop.mapreduce.Job ;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat ;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat ;