Database Reference
In-Depth Information
The Spade system (Stream Processing Application Declarative Engine) [10] is
the declarative stream processing engine of System S. It provides a rapid application
development front end for System S using the following features:
An intermediate language for flexible composition of parallel and distrib-
uted data-flow graphs. This language sits in between higher-level program-
ming tools and languages such as the System S IDE or Stream QL, and the
lower-level System S programming APIs.
A toolkit of type generic built-in stream processing operators. Spade supports
all  basic stream-relational operators with rich windowing and punctuation
semantics. It also seamlessly integrates built-in operators with user-defined ones.
A broad range of stream adapters. These adapters are used to ingest data
from outside sources and publish data to outside destinations, such as net-
work sockets, relational and XML databases and file systems.
The Spade language provides a stream-centric, operator-level programming
model. The stream-centric design implies building a programming language where
the basic building block is a stream. The operators can be used to implement any rela-
tional query with windowing extensions used in streaming applications. Examples of
these operators are
Source , used for creating a stream from data flowing from an external
source. This operator is capable of performing parsing and tuple creation
and can interact with a diverse set of external devices.
Sink , used for converting a stream into a flow of tuples that can be used by
components that are not part of System S. Its main task consists of convert-
ing tuples into objects accessible externally through devices such as the file
system or the network.
Functor , used for performing tuple-level manipulations such as filtering,
projection, mapping, attribute creation, and transformation. In these manipu-
lations, the Functor operator can access tuples that have appeared earlier in
the input stream.
Aggregate , used for grouping and summarization of incoming tuples. This
operator supports a large number of grouping mechanisms and summariza-
tion functions.
Join , used for correlating and pairing two streams.
Sort , used for imposing an order on incoming tuples in a stream.
Barrier , used as a synchronization point that consumes tuples from mul-
tiple streams, outputting a tuple only when a tuple from each of the input
streams has arrived.
Punctor , used for performing tuple-level manipulations where conditions
on the current tuple as well as on past tuples are evaluated for generating
punctuations in the output stream.
Split , used for routing incoming tuples to different output streams based on
a user-supplied routing condition.
Delay , used for delaying a stream based on a user-supplied time interval.
Search WWH ::




Custom Search