Database Reference
In-Depth Information
Filter
The
filter
operator is similar to a filter in an actual water stream, or in
your furnace or car: its purpose is to allow only some of the streaming con-
tents to pass. A Streams
filter
operator removes tuples from a data stream
based on a user-defined condition specified as a parameter to the operator.
After you've programmatically specified a condition, the first output port
defined in the operator will receive any tuples that satisfy that condition. You
can optionally specify a second output port to receive any tuples that did not
satisfy the specified condition. (If you're familiar with extract, transform, and
load [ETL] flows, this is similar to a
match
and
discard
operation.)
Functor
The
functor
operator reads from an input stream, transforms tuples in flex-
ible ways, and sends new tuples to an output stream. The transformations
can manipulate any of the elements in the stream. For example, you could
extract a data element out of a stream and output the running total of that
element for every tuple that comes through a specific
functor
operator.
Punctor
The
punctor
operator adds punctuation into the stream, which can then be
used downstream to separate the stream into multiple windows. For exam-
ple, suppose a stream reads a contact directory listing and processes the data
flowing through that stream. You can keep a running count of last names in
the contact directory by using the
punctor
operator to add a punctuation
mark into the stream any time your application observes a changed last name
in the stream. You could then use this punctuation mark downstream in an
aggregation
functor
operator to send out the running total for that name,
later resetting the count back to
0
to start counting the next set of last names.
Sort
The
sort
operator outputs the tuples that it receives, but in a specified sorted
order. This operator uses a stream window specification. Think about it for a
moment: If a stream represents a constant flow of data, how can you sort that
data? You don't know whether the next tuple to arrive will need to be sorted
with the first tuple to be sent as output. To overcome this issue, Streams
enables you to specify a window on which to operate. You can specify a win-
dow of tuples in the following ways:
