Database Reference
In-Depth Information
7.5.7 Conclusions and Future Work
Our experiences with MonetDB/SkyServer application confirm the advan-
tages of column-wise storage systems for scientific applications with analyt-
ical disk-bound processing. To improve the performance for point-and-range
queries several techniques for workload-driven self-organization of columns
have been developed in MonetDB, such as cracking (continuous physical or-
ganization based on access patterns),
61
and adaptive segmentation and repli-
cation (splitting columns into segments or replicating segments).
62
We in-
tend to integrate those techniques in support of the SkyServer application.
Since compression has shown to be particularly ecient in combination with
column-wise storage,
18
we also intend to investigate and utilize appropriate
compression schemes for the SkyServer application.
The MonetDB execution engine differs in a fundamental way from state-
of-the-art commercial systems. The execution paradigm is based on full ma-
terialization of all intermediate results in a query plan. This opens another
direction of research exploiting commonalities in query batches by carefully
preserving and reusing common intermediate results.
7.6 Extremely Large Databases and SciDB
In this section we describe very recent developments in the area of scientific
databases that may lead to yet another type of database architecture that is
neither horizontal nor vertical but rather based on array structures. These de-
velopments were initiated in two successive workshops called Extremely Large
Databases (XLDB)
63
,
64
that were organized in order to address the challenge
of designing databases that can support the complexity and scale involved
in scientific applications. An important outcome of these workshops was the
foundation of an organization consisting of researchers and implementers from
a variety of disciplines dedicated to the design and implementation of a new
open source science database called SciDB.
7.6.1 Differences between the Requirements of Scientific
and Commercial Databases
Major differences between the requirements of large scientific databases and
current commercial DBMS offerings were noted.
65
This led to the conclusion
that the new database system should not just consist of incremental improve-
ments to existing commercial DBMSs but requires a complete new design
from the ground up. The most important differences between these two types
of databases are summarized in Table 7.2.
