Database Reference
In-Depth Information
The
Spade
system (Stream Processing Application Declarative Engine) [10] is
the declarative stream processing engine of System S. It provides a rapid application
development front end for System S using the following features:
•
An intermediate language for flexible composition of parallel and distrib-
uted data-flow graphs. This language sits in between higher-level program-
ming tools and languages such as the System S IDE or Stream QL, and the
lower-level System S programming APIs.
•
A toolkit of type generic built-in stream processing operators. Spade supports
all basic stream-relational operators with rich windowing and punctuation
semantics. It also seamlessly integrates built-in operators with user-defined ones.
•
A broad range of stream adapters. These adapters are used to ingest data
from outside sources and publish data to outside destinations, such as net-
work sockets, relational and XML databases and file systems.
The Spade language provides a stream-centric, operator-level programming
model. The stream-centric design implies building a programming language where
the basic building block is a stream. The operators can be used to implement any rela-
tional query with windowing extensions used in streaming applications. Examples of
these operators are
•
Source
, used for creating a stream from data flowing from an external
source. This operator is capable of performing parsing and tuple creation
and can interact with a diverse set of external devices.
•
Sink
, used for converting a stream into a flow of tuples that can be used by
components that are not part of System S. Its main task consists of convert-
ing tuples into objects accessible externally through devices such as the file
system or the network.
•
Functor
, used for performing tuple-level manipulations such as filtering,
projection, mapping, attribute creation, and transformation. In these manipu-
lations, the Functor operator can access tuples that have appeared earlier in
the input stream.
•
Aggregate
, used for grouping and summarization of incoming tuples. This
operator supports a large number of grouping mechanisms and summariza-
tion functions.
•
Join
, used for correlating and pairing two streams.
•
Sort
, used for imposing an order on incoming tuples in a stream.
•
Barrier
, used as a synchronization point that consumes tuples from mul-
tiple streams, outputting a tuple only when a tuple from each of the input
streams has arrived.
•
Punctor
, used for performing tuple-level manipulations where conditions
on the current tuple as well as on past tuples are evaluated for generating
punctuations in the output stream.
•
Split
, used for routing incoming tuples to different output streams based on
a user-supplied routing condition.
•
Delay
, used for delaying a stream based on a user-supplied time interval.
Search WWH ::
Custom Search