Database Reference
In-Depth Information
network by the WaveScope runtime system. Internally, this binary consists of
a sequence of operators similar to that used by conventional DSMSs.
11.2.6 The WaveScope Runtime
This compiled binary is fed to the WaveScope Runtime for execution. As
with other stream-processing engines described above, operators in compiled
WaveScript query plans are run by a scheduler, which picks boxes one at a
time and runs them to produce outputs.
A special
timebase manager
is responsible for managing timing informa-
tion corresponding to signal data. This is a common problem in signal pro-
cessing applications, since signal processing operators typically process vec-
tors of samples with sequence numbers, leaving the application developer
to determine how to interpret those samples temporally. A timebase is a
dynamic data structure that represents and maintains a mapping between
sample sequence numbers and time units. Examples of timebases include
those based on abstract units (such as seconds, hertz, and meters), as well
as timebases based on real-world clocks, such as a CPU counter or sample
number.
Finally, a
memory manager
is responsible for creation, storage, and man-
agement of memory, particularly as is associated with SigSeg objects that
represent the majority of signal data in WaveScope. Designing an ecient
memory manager is of critical importance, since memory allocation can con-
sume significant processing time during query processing.
These features—the timebase manager, memory manager, and scheduler—
simplify the task of the programmer (who no longer has to worry about these
details) and allow WaveScope to provide excellent performance. In our initial
benchmarks of the marmot application, the current WaveScope system pro-
cesses 7 million samples per second on a 3 GHz PC, and about 80 K samples
per second on a 400 MHz ARM platform (which has a 10x penalty for floating
point emulation in addition to a reduced processor speed).
11.2.7 Conclusion
Data stream management systems provide an ecient and powerful way to
process data arriving in near real time and are well suited to applications rang-
ing from financial analysis to network monitoring. Scientific applications, how-
ever, present an “impedance mismatch” as they often involve large amounts
of custom signal processing code and also require support for very high data
rates. The WaveScope DSMS, and associated WaveScript language, are de-
signed especially to support such applications, including an integrated lan-
guage for writing data and signal processing operations, an ecient way to
represent batches of high data rate, isochronous signals as SigSegs, mecha-
nisms for transmitting data between networked stream processing nodes, and
