Database Reference
In-Depth Information
Local or shuffle grouping
Local or shuffle grouping is the most common grouping that randomly distributes the
tuples emitted by the source ensuring equal distribution, that is, each instance of the bolt
gets to process the same number of events. Load balancing is automatically taken care of
by this grouping.
Due to the random nature of distribution of this grouping, it's useful only for atomic opera-
tions by specifying a single parameter—source of stream. The following snippet is from
WordCount
topology (which we reated earlier), which demonstrates the usage of shuffle
grouping:
TopologyBuilder myBuilder = new TopologyBuilder();
builder.setSpout("spout", new RandomSentenceSpout(), 5);
builder.setBolt("split", new SplitSentence(),
8).shuffleGrouping("spout");
builder.setBolt("count", new WordCount(),
12).fieldsGrouping("split", new Fields("word"));
In the following figure, shuffle grouping is depicted:
Here
Bolt A
and
Bolt B
both have a parallelism of two, each; so two instances of each of
these bolts is spawned by the Storm framework. These bolts are wired together by
shuffle
grouping
. We will now discuss the distribution of events.
The 50 percent events from
Instance 1
of
Bolt A
would go to
Instance 1
of
Bolt B
, and
the remaining 50 percent would go to
Instance 2
of
Bolt B
. Similarly, 50 percent of events
emitted by
Instance 2
of
Bolt B
would go to
Instance 1
of
Bolt B
, and the remaining 50
percent would go to
Instance 2
of
Bolt B
.
Search WWH ::
Custom Search