Database Reference
In-Depth Information
scientific data. The system has been applied to several scientific signal
processing applications. The importance here is the use of streams as
first-class language objects and ecient representations of windows and
subwindows over streams. WaveScope provides a typed functional pro-
gramming language where high-volume signal processing can be specified
through the signal segment datatype. Several distributed WaveScope nodes
can communicate over distributed stream communication channels spec-
ified by the user. The section includes a comparison of this approach
with DSMSs and discusses the advantage of using a functional program-
ming language for high-volume streams, which traditional DSMSs do not
provide.
Section 11.3 describes approaches to specify massively parallel scientific
data stream queries. For scalability of stream-processing algorithms it is nec-
essary to provide query language primitives describing how to parallelize a
stream computation. It is not always possible to provide fully automated and
transparent distribution. Therefore, the query language needs primitives al-
lowing the user to parallelize computations and filters. The second section
describes two approaches to specify parallel stream processing: data flow dis-
tribution templates and stream processes in the GSDM (Grid Stream Data
Manager) and SCSQ (Super Computer Stream Query processor) systems, re-
spectively. Both systems are based on a functional data model and query
language. 9
Finally, Section 11.4 discusses how to speed up streaming computations
of functions by approximate materialization of computed values for scien-
tific simulations. Often, queries over streaming scientific data involve complex
computations expressed as functions. These functions may be costly to ex-
ecute. Therefore, approximate materializations and indexing of their results
may speed up the processing by avoiding recomputations.
11.2 Stream Processing for Scientific Applications
DSMSs are ideally suited for online scientific data processing applications,
because they provide:
1. Windowed operations: The ability to segment incoming data streams
into windows , which can then be sorted or aggregated to compute statis-
tics such as the mean of some element in the stream.
2. Main memory operation: The ability to process data without first send-
ing it to disk, decreasing latency and improving throughput.
3. Specialized streaming operators: For example, operators that detect
out-of-order stream elements or that detect sequences or patterns in
streams.
Search WWH ::




Custom Search