Database Reference
In-Depth Information
The Supervisors
Supervisors in Storm are similar to
TaskTrackers
in Hadoop. They are
servers that run on each of the worker nodes and manage the execution of
local tasks. Each of the local tasks is either a spout or a bolt, and there can
be many local tasks running under the control of each supervisor.
During normal operation, if the supervisor goes down then all of the tasks
running under its control are also lost. Because Storm is designed to be
distributed and fault tolerant, when this happens the Nimbus restarts these
tasks under the control of a supervisor.
When a supervisor returns to a cluster (or more supervisors are added),
running topologies are not automatically rebalanced. To rebalance the
cluster after adding more nodes, the
rebalance
command should be run
from the Storm command-line interface.
Configuring a Storm Cluster
This section demonstrates how to configure a Storm cluster for distributed
mode processing. On distributed hardware, this would be the mode for
production processing. It is also possible to run all services on a single
machine, although it is often easier to use the local cluster mode described
in the next section for development and debugging.
Prerequisites
To begin, obtain a Storm binary archive from
http://storm-project.net
website
. The most recent version at the time of writing is 0.9.0.1, and this is
the version recommended for installation.
If, for some reason, it is necessary to use an older version of Storm, you
need to install ZeroMQ (also written 0MQ). This restriction is removed in
0.9.0 thanks to the Netty transport, but earlier versions of Storm require
this package for nodes to be able to communicate with each other.
OnLinuxsystems,ZeroMQisoftenavailable fromthepackagemanagement
system, but you should take care to install version 2.1.7. This is the only
version that is known to work with Storm. Mac OS X users can obtain 0MQ
from package libraries such as Homebrew or MacPorts, but Windows users
have many more problems. It is easier to simply upgrade to Storm 0.9.0
than it is to make an older version of Storm work on Windows.
