Database Reference
In-Depth Information
Back-end
Cluster
Blue
Gene
Front-end
Cluster
User
Figure 11.9
Stream data flow in the LOFAR environment.
output is sent to
a
and
b
. Using postfilters, the query above is transformed
into the following query:
create function radix2(String ss) -> Stream
as select radixcombine(merge({a,b}))
from SP a, SP b, SP c
where a=sp(fft(extract(c)))
and b=sp(fft(extract(c)))
and c=sp(receiver(ss), #'oddeven');
The notation
#'oddeven'
specifies the object representing the function named
oddeven
. This function is the postfilter function that extracts odd elements
of the vector when sent to
a
and even elements when sent to
b
. Postfilter
functions have successfully been used in spatial partitioning for parallelization
of combinatorial optimization problems.
24
Figure 11.9 illustrates the stream dataflow in the LOFAR hardware envi-
ronment. Users interact with SCSQ on a Linux front-end cluster. Another
Linux back-end cluster first receives the streams from the sensors where they
are preprocessed. Next, the BlueGene processes these streams. The output
streams from the BlueGene are then postprocessed in the front-end cluster,
and the result stream is finally delivered to the user. Thus, three computer
clusters are involved.
Figure 11.10 illustrates a query that is set up for execution in the hardware
environment. SCSQ users interact with the client manager, in which they
specify CQs using SCSQL. The execution of a CQ forms a directed acyclic
graph of running processes (RPs), each executing the subquery specified in
one SP.
Back-end
BlueGene
Front-end
RP
RP
Client
Manager
RP
RP
RP
RP
RP
beCC
bgCC
feCC
RP
RP
CNDB
CNDB
CNDB
Figure 11.10
Parallel CQ execution in SCSQ. Wide arrows indicate data
streams.
