Database Reference
In-Depth Information
@Override
public int compareTo ( TextPair tp ) {
int cmp = first . compareTo ( tp . first );
if ( cmp != 0 ) {
return cmp ;
}
return second . compareTo ( tp . second );
}
}
The first part of the implementation is straightforward: there are two Text instance vari-
ables, first and second , and associated constructors, getters, and setters. All Writ-
able implementations must have a default constructor so that the MapReduce framework
can instantiate them, then populate their fields by calling readFields() . Writable
instances are mutable and often reused, so you should take care to avoid allocating objects
in the write() or readFields() methods.
TextPair 's write() method serializes each Text object in turn to the output stream
by delegating to the Text objects themselves. Similarly, readFields() deserializes
the bytes from the input stream by delegating to each Text object. The DataOutput
and DataInput interfaces have a rich set of methods for serializing and deserializing
Java primitives, so, in general, you have complete control over the wire format of your
Writable object.
Just as you would for any value object you write in Java, you should override the
hashCode() , equals() , and toString() methods from java.lang.Object .
The hashCode() method is used by the HashPartitioner (the default partitioner
in MapReduce) to choose a reduce partition, so you should make sure that you write a
good hash function that mixes well to ensure reduce partitions are of a similar size.
WARNING
If you plan to use your custom Writable with TextOutputFormat , you must implement its
toString() method. TextOutputFormat calls toString() on keys and values for their output
representation. For TextPair , we write the underlying Text objects as strings separated by a tab char-
acter.
TextPair is an implementation of WritableComparable , so it provides an imple-
mentation of the compareTo() method that imposes the ordering you would expect: it
sorts by the first string followed by the second. Notice that, apart from the number of
Search WWH ::




Custom Search