Database Reference
In-Depth Information
A Round-Robin Custom Stream Grouping
This example implements a round-robin mechanism for distributing
data among bolt tasks. It is no more efficient than a
shuffleGrouping , but it demonstrates the technique.
First, the class is defined with a serialization version number. Storm
makes heavy use of Java serialization when configuring a topology, so it
is important to pay attention to what will be serialized and what will
not. The three class fields are all set during the prepare statement so
they should be defined as transient so they will not be included in
the serialization:
public class RoundRobinGrouping implements
CustomStreamGrouping {
/**
*
*/
private static final long serialVersionUID = 1L;
transient List<Integer> targetTasks;
transient int
nextTask;
transient int
numTasks;
Next, the prepare statement initializes each of the transient variables.
A more complicated grouping might inspect the topology itself to assign
weightings to tasks and other options, but this simple round-robin
method just needs to hold onto the list:
public void prepare(WorkerTopologyContext context,
GlobalStreamId stream,
List<Integer> targetTasks) {
this .targetTasks = targetTasks;
this .numTasks = targetTasks.size();
this .nextTask = 0;
}
Search WWH ::




Custom Search