Database Reference
In-Depth Information
to the query. The CPU cores shown in the figure also run in parallel and pro-
cess the remaining data in the stream concurrently. We kept this figure simple
to make it easy to read, but the process described here is repeated on about
one hundred of these parallel snippet processors running in the appliance.
On a ten-rack system, this would represent up to one thousand parallel snip-
pet processors, with performance that exceeds that of much more expensive
systems by orders of magnitude.
The FPGA includes the following embedded engines:
•
Thecompressionengine
All data in Netezza is stored compressed
on disk. The compression engine resides inside the FPGA and
decompresses data at the transmission speed of the network, instantly
transforming each block on disk into 4 to 8 blocks in memory. The use
of semiconductor-based technology, and not software, to achieve
compression and decompression of data boosts system performance
by a factor of 4 to 8, which results in significant acceleration of the
slowest component in any data warehouse, the disk.
•
Theprojectandrestrictengine
This engine further enhances
performance by filtering out columns and rows, based on parameters
in the
SELECT
and
WHERE
clauses of an SQL query.
•
Thevisibilityengine
Netezza maintains ACID (atomicity,
consistency, isolation, and durability) compliance at streaming speeds,
and uses its visibility engine to filter out rows that should not be
“seen” by a query (for example, rows belonging to a transaction that's
not yet committed).
Query Orchestration in Netezza
All queries and analyses enter Netezza through the host machine where the
optimizer, the compiler, and the scheduler decompose them into many dif-
ferent snippets whose instructions are subsequently distributed into the mul-
tiple S-blades (all of which then process their workload simultaneously
against their locally managed slice of data). In this section, we discuss these
individual steps in greater detail using Figure 4-4.
Query Optimization
The intelligence of the Netezza optimizer is one of its greatest strengths. It
was engineered to create query execution plans specifically optimized for
