Database Reference
In-Depth Information
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
job.setInputFormatClass(ColumnFamilyInputFormat.class);
FileOutputFormat.setOutputPath(job, new Path(“/tmp/word_count”));
ConfigHelper.setThriftContact(job.getConfiguration(), “localhost”, 9160);
ConfigHelper.setInputColumnFamily(
job.getConfiguration(), “Keyspace1”, “Standard1”);
SlicePredicate predicate = new SlicePredicate().setColumn_names(
Arrays.asList(columnName.getBytes()));
ConfigHelper.setInputSlicePredicate(job.getConfiguration(), predicate);
job.waitForCompletion(true);
return 0;
}
}
There are a few things to note about the
WordCount
class that go beyond a boilerplate word
count. We need to set the
InputFormat
to Cassandra's
ColumnFamilyInputFormat
, along
with the column name that our mapper is looking for. Cassandra includes something called a
ConfigHelper
that provides a way to set properties we'll need, such as the Thrift contact in-
formation (server and port), the keyspace, and the column family. It also allows us to set our slice
predicate.
Outputting Data to Cassandra
Our example reducer used a
FileOutputFormat
to output its results. As its name suggests,
FileOutputFormat
writes the results out to a filesystem. As of 0.7, there will be a Cassandra-
based
OutputFormat
implementation. As of this writing, some of the implementation details
are still being finalized. For updates on the built-in output format, see
http://wiki.apache.org/cas-
It is possible, however, to write directly to Cassandra via Thrift (or a higher-level client) in the
Reducer
step. In the previous example, this means that instead of writing to the context, one
could write its key and value to Cassandra directly.
Hadoop Streaming
Our word count MapReduce example was written in Java. Hadoop Streaming is the Hadoop way
to allow languages besides Java to run MapReduce jobs using Standard In and Standard Out.
As of this writing, Hadoop Streaming is not supported in Cassandra's MapReduce integration.
