MapReduce Features - Hadoop: The Definitive Guide - page 295

Counter Description

for measuring core and memory usage ( VCORES_MILLIS_REDUCES
and MB_MILLIS_REDUCES ).

User-Defined Java Counters

MapReduce allows user code to define a set of counters, which are then incremented as
desired in the mapper or reducer. Counters are defined by a Java enum, which serves to
group related counters. A job may define an arbitrary number of enums, each with an ar-
bitrary number of fields. The name of the enum is the group name, and the enum's fields
are the counter names. Counters are global: the MapReduce framework aggregates them
across all maps and reduces to produce a grand total at the end of the job.
We created some counters in Chapter 6 for counting malformed records in the weather
dataset. The program in Example 9-1 extends that example to count the number of miss-
ing records and the distribution of temperature quality codes.

Example 9-1. Application to run the maximum temperature job, including counting missing
and malformed fields and quality codes
public class MaxTemperatureWithCounters extends Configured implements
Tool {

enum Temperature {
MISSING ,
MALFORMED

}

static class MaxTemperatureMapperWithCounters
extends Mapper < LongWritable , Text , Text , IntWritable > {

private NcdcRecordParser parser = new NcdcRecordParser ();

@Override
protected void map ( LongWritable key , Text value , Context context )
throws IOException , InterruptedException {

parser . parse ( value );
if ( parser . isValidTemperature ()) {
int airTemperature = parser . getAirTemperature ();
context . write ( new Text ( parser . getYear ()),
new IntWritable ( airTemperature ));
} else if ( parser . isMalformedTemperature ()) {
System . err . println ( "Ignoring possibly corrupt input: " + value );
context . getCounter ( Temperature . MALFORMED ). increment ( 1 );