An Overview of Large-Scale Stream Processing Engines - Large Scale and Big Data: Processing and Management - page 392

Inputs Outputs

Storage
manager

Router

…
…

Q 1
Q 2

σ
µ

.

Scheduler .

…

Q i

Buffer manager

Box processors

Catalogs

store

Persistent

…

Q j

.

Load
shedder

QoS
monitor

…

Q n

FIGURE 12.2 The runtime architecture of Aurora.

12.3 BOREALIS
Borealis [1,6,12] is one of the early presented distributed stream processing engines.
In Borealis, the collection of continuous queries submitted to the system can be
seen as one giant network of operator query diagram whose processing is distrib-
uted to multiple sites. Each site runs a Borealis server where the Query Processor
(QP) is a single-site processor. Figure 12.3 illustrates the system architecture of
Borealis where input streams are fed into the QP an results are pulled through I/O
Queues , which route tuples to and from remote Borealis nodes and clients. The QP
is controlled by the Admin module that sets up locally running queries and takes
care of moving query diagram fragments to and from remote Borealis nodes, when
instructed to do so by another module. System control messages issued by the Admin
are fed into the Local Optimizer . Local Optimizer further communicates with major
run-time components of the QP to supply them with performance improving direc-
tions. These components are

• Priority scheduler , which determines the order of box execution based on

tuple priorities.

• Box processors , with one for each different type of box, which can change

behavior on the fly based on control messages from the local optimizer.

• Load shedder , which discards low-priority tuples when the node is

overloaded.

The QP also contains the storage manager , which is responsible for the storage
and retrieval of data that flows through the arcs of the local query diagram. The local
catalog stores query diagram descriptions and metadata, which are accessible by all
the components.