Database Reference
In-Depth Information
topology to achieve higher performance by combining several operations
into a single physical bolt and avoiding the communication overhead
between bolts.
This section covers using this high-level construct to build topologies in
Storm.
Trident Streams and Spouts
Trident topologies usually operate on a
Stream
object. The stream in
Trident is conceptually very similar to the stream defined by a spout or a
bolt in a normal Storm topology. The difference here is that the stream's
structure is defined and modified in the topology definition rather than
being an intrinsic part of a bolt.
Defining the stream structure along with the topology has a couple of
advantages when writing topologies. First, by separating the tuple structure
from the business logic, it is easier to write reusable components. Second,
the topology becomes much easier to read. Because the data flow and data
definition is in the same place, it is much easier to see the manipulations.
In “traditional” topologies, determining how each stream is defined involves
inspecting each
Bolt
individually. The disadvantage is that it can be harder
to express computations using Trident's language than it would be to
explicitly define the topology. Often this is simply a matter of preference.
Streams in Trident topologies are built around a
Spout
. These spouts can
be the same
IRichSpout
implementations from standard topologies, or
they can be one of the more specialized Trident spouts. Using a basic spout,
such as the
LoremIpsumSpout
, is as simple as defining a stream using
newStream
in the
TridentTopology
class:
TridentTopology topology =
new
TridentTopology();
topology.newStream("lorem",
new
LoremIpsumSpout());
Local Operations: Filters and Functions
When constructing topologies, Trident considers certain operations to be
“partition local” operations. What this means is that a chain of such
operations can be executed locally within a single bolt. This enables Trident
to make use of more efficient data structures and reduce the amount of
communication required.
