Database Reference
In-Depth Information
Guaranteed processing of the Storm cluster
The next topic to discuss in this section is to see
Storm's guaranteed message processing in
action
. We discussed this concept in previous chapters, but to understand it practically, I
didn't go into depth because I wanted to introduce you all to the AMQP spout first. Now
let's go back to the example we discussed in
Chapter 2
,
Getting Started with Your First To-
pology
.
Now as depicted in the following figure, the dash arrow flow shows that the events that fail
to process are re-queued to the queue:
Now let's tweak our
wordCount
topology a bit where we had added
AMQPRecvSpout
to fail the events, and see where they actually show up. Let's assume I used
FixedEmit-
ter
to emit 10 events into the queue. Now I tweak my
wordCount
bolt and induce artifi-
cial sleep for five minutes in the execute method, so that every event is held there for 300
seconds (using
Thread.sleep(300)
). This will lead to its timeout as the default event
timeout is 60 seconds.
Now when you run the topology, you will be able to see the events being re-queued back to
RabbitMQ using the UI.
Search WWH ::
Custom Search