Accelerating Queries on Very Large Datasets - Scientific Data Management - page 184

6.7.2 Indexes Still Useful ........................................... 221

6.7.3 Using Indexes to Make Smart Iterators ...................... 224

6.8 Summary and Future Trends ........................................ 226

Acknowledgment ........................................................... 227

References ................................................................. 227

6.1 Introduction
One of the primary goals of a data management system (DBMS) is to retrieve
the records under its control upon user requests. In the SQL language, such
retrievals are typically formed as queries. Answering these queries eciently is
a key design objective of a data management system. To achieve this goal, one
needs to consider many issues including data organization, available methods
for accessing the data, user interface, effective query execution planning, and
the overall system design. In this chapter, we primarily focus on the aspects
that have the most direct influence on the eciency of query processing, which
primarily include three of them: the data organization, access methods, and
query execution planning. 25 , 39 , 47 , 67 , 79 , 99
Usually a query can be answered in different ways, for example, the tables
and columns involved may be retrieved in different orders or through different
access methods. 23 , 44 , 81 However, these choices are built on top of a set of good
access methods and data organizations. Therefore, we choose to concentrate
more on the issues of data organizations and access methods. Furthermore,
many common types of queries on scientific data do not require complex exe-
cution plans, as we explain in the next section. Thus, optimizing the execution
plan is less important than the other two issues. Furthermore, much of the
scientific data is not under the control of a DBMS system, but is under the
control of some stand-alone systems or emerging scientific DBMSs. A discus-
sion on the core data access methods and data organizations may influence
the design and implementation of such systems.
On the issue of data organization, a fundamental principle of the database
research is the separation of logical data organization from the physical data
organization. Since most DBMSs are based on software that does not have
direct control of the physical organization on secondary storage systems, we
primarily concentrate on logical organization in this chapter. One common
metaphor of logical data organization is the relational data model, consisting
of tables with rows and columns. Sometimes, a row is also called a tuple, a
data record, or a data object; and a column is also known as an attribute of
a record, or a variable in a dataset.
There are two basic strategies of partitioning a table: the row-oriented
organization that places all columns of a row together, and the column-
oriented organization that places all rows of a column together. The row-
oriented organization is also called the horizontal data organization, while the