Database Reference
In-Depth Information
NullWritable
NullWritable
is a special type of Writable, as it has a zero-length serialization. No
bytes are written to or read from the stream. It is used as a placeholder; for example, in
MapReduce, a key or a value can be declared as a
NullWritable
when you don't need
to use that position, effectively storing a constant empty value.
NullWritable
can also
be useful as a key in a
SequenceFile
when you want to store a list of values, as op-
posed to key-value pairs. It is an immutable singleton, and the instance can be retrieved by
calling
NullWritable.get()
.
ObjectWritable and GenericWritable
ObjectWritable
is a general-purpose wrapper for the following: Java primitives,
String
,
enum
,
Writable
,
null
, or arrays of any of these types. It is used in Hadoop
RPC to marshal and unmarshal method arguments and return types.
ObjectWritable
is useful when a field can be of more than one type. For example, if
the values in a
SequenceFile
have multiple types, you can declare the value type as an
ObjectWritable
and wrap each type in an
ObjectWritable
. Being a general-pur-
pose mechanism, it wastes a fair amount of space because it writes the classname of the
wrapped type every time it is serialized. In cases where the number of types is small and
known ahead of time, this can be improved by having a static array of types and using the
index into the array as the serialized reference to the type. This is the approach that
Gen-
ericWritable
takes, and you have to subclass it to specify which types to support.
Writable collections
The
org.apache.hadoop.io
package includes six
Writable
collection types:
ArrayWritable
,
ArrayPrimitiveWritable
,
TwoDArrayWritable
,
MapWritable
,
SortedMapWritable
, and
EnumSetWritable
.
ArrayWritable
and
TwoDArrayWritable
are
Writable
implementations for ar-
rays and two-dimensional arrays (array of arrays) of
Writable
instances. All the ele-
ments of an
ArrayWritable
or a
TwoDArrayWritable
must be instances of the
same class, which is specified at construction as follows:
ArrayWritable writable
=
new
ArrayWritable
(
Text
.
class
);
In contexts where the
Writable
is defined by type, such as in
SequenceFile
keys or
values or as input to MapReduce in general, you need to subclass
ArrayWritable
(or
TwoDArrayWritable
, as appropriate) to set the type statically. For example:
