Database Reference
In-Depth Information
Figure 24-4. A simple PipeAssembly
On the surface, this might seem more complex than the traditional MapReduce model.
And admittedly, there are more concepts here than map, reduce, key, and value. But in
practice, there are many more concepts that must all work in tandem to provide different
behaviors.
For example, a developer who wanted to provide a “secondary sorting” of reducer values
would need to implement a map, a reduce, a “composite” key (two keys nested in a parent
key), a value, a partitioner, an “output value grouping” comparator, and an “output key”
comparator, all of which would be coupled to one another in varying ways, and very
likely would not be reusable in subsequent applications.
In Cascading, this would be one line of code:
new GroupBy(
<previous>
,
<grouping fields>
,
<secondary sorting fields>
)
, where
<previ-
ous>
is the pipe that came before.
