Database Reference
In-Depth Information
A Round-Robin Custom Stream Grouping
This example implements a round-robin mechanism for distributing
data among bolt tasks. It is no more efficient than a
shuffleGrouping
, but it demonstrates the technique.
First, the class is defined with a serialization version number. Storm
makes heavy use of Java serialization when configuring a topology, so it
is important to pay attention to what will be serialized and what will
not. The three class fields are all set during the
prepare
statement so
they should be defined as
transient
so they will not be included in
the serialization:
public class
RoundRobinGrouping
implements
CustomStreamGrouping {
/**
*
*/
private static final long
serialVersionUID
= 1L;
transient
List<Integer> targetTasks;
transient int
nextTask;
transient int
numTasks;
Next, the
prepare
statement initializes each of the transient variables.
A more complicated grouping might inspect the topology itself to assign
weightings to tasks and other options, but this simple round-robin
method just needs to hold onto the list:
public void
prepare(WorkerTopologyContext context,
GlobalStreamId stream,
List<Integer> targetTasks) {
this
.targetTasks = targetTasks;
this
.numTasks = targetTasks.size();
this
.nextTask = 0;
}