Database Reference
In-Depth Information
Distributed Remote Procedure Calls (DRPC), which are used to easily
implement distributed processing services, such as performing a large
number of expensive calculations. Using a special spout and bolt
combination, these topologies implement a complicated procedure that can
be distributed across multiple machines.
This is not strictly something that is typically done in a real-time streaming
analysis scenario, but it can certainly be used to scale parts of a real-time
analysis pipeline. It can also be used to build component-based services that
make an environment easier to manage.
The DRPC Server
DRPC requests are managed by a separate Storm server that manages the
communication between the DRPC client and the Storm topology servicing
those requests. In a distributed environment, this server is started with the
Storm command-line utility:
$ ./bin/storm drpc
In testing environments, a Local DRPC server can be started. This is similar
to the
LocalCluster
option used for testing:
TopologyBuilder builder =
new
TopologyBuilder();
LocalDRPC drpc =
new
LocalDRPC();
Writing DRPC Topologies
A DRPC topology is just like any other Storm topology, relying on a special
Spout
and
Bolt
combination to enact the DRPC functionality. The
topology takes in a single
String
value via the Spout and returns a single
String
value via the
ReturnResults
bolt. For example, the following
topology implements a “power” function that exponentiates a
comma-separated list of doubles corresponding to the base value and the
exponent. It returns a string representing the power output:
//Omit the drpc argument when submitting to the cluster
builder.setSpout("drpc",
new
DRPCSpout("power", drpc));
builder.setBolt("split",
new
SplitBolt()).shuffleGrouping("drpc");
builder.setBolt("power",
new
