Database Reference
In-Depth Information
After briefly reviewing traditional DSMSs, we focus on scientific applica-
tions and the
WaveScope
scientific stream processing system that has been
developed at MIT over the past few years.
11.2.1 Stream Processing Background
The DSMSs
3
-
8
include a common feature set: a high-level, declarative or
graphical programming interface that allows data streams to be filtered ac-
cording to some set of simple predicates, combined with each other (“joined”)
to match up related elements by time or values, and “aggregated” to compute
simple statistics over groups of tuples.
Traditional applications of DSMSs cover a range of applications, from net-
work monitoring to financial tick stream analysis. For example, suppose a user
wants to find times when the price of IBM goes up for three stock ticks in a
row. This could be expressed through the following “StreamSQL” query (this
example is based on one given at http://www.streambase.com/developers-
library-articles-detectingpatterns.htm):
CREATE INPUT STREAM InputStream
(stock string(5), price double, time timestamp);
SELECT s1.time,s1.price AS price1
FROM PATTERN InputStream AS s1 THEN
InputStream AS s2 THEN
InputStream AS s3
WITHIN 20 TIME
WHERE s1.stock=s2.stock AND s2.stock=s3.stock AND
s3.price>s1.price AND s1.price>s2.price AND
s1.stock = 'IBM';
The first statement defines a stream
InputStream
. The
SELECT
statement
then looks for a sequence of three stocks that satisfy the predicates in the
WHERE
clause (consecutively increasing, and all with stock symbol “IBM”)
and which arrive within 20 seconds of each other, and returns the time and
price of the first stock in such a pattern.
Another simple query over this data stream might compute the average
price of IBM stocks in the last hour, every minute, assuming one quote ar-
rives pe minute:
SELECT time, AVG(price)
FROM InputStream [SIZE 60 ADVANCE 1] AS s
WHERE stock = 'IBM'
Here, the
[]
notation indicates that the incoming stream should be “win-
dowed” into batches of 60 tuples, and that each consecutive window should
include one new tuple and discard one old tuple.
