Database Reference
In-Depth Information
4. The processor core picks up the data stream and performs core
database operations such as sorts, joins, aggregations, and so on. It
also applies complex algorithms that are embedded in the snippet
processor for advanced analytics. Results of each snippet processor's
work are assembled in memory to produce a subresult for the entire
snippet. With more than a thousand snippet processors working
simultaneously, hundreds or thousands of query snippets can
execute in parallel. Snippet processors use the intelligent network
fabric to communicate with the host (and with each other) and
perform intermediate calculations and aggregations. Some highly
complex algorithms, such as matrix computations, require
communication among nodes. For this reason, Netezza engineered a
message-passing interface to communicate interim results and to
produce the final result.
5. Finally, the host assembles the intermediate results received from
the snippet processors, compiles the final result set, and returns it
to the application. Of course, during this time, other queries are
streaming through the system at various stages of completion.
This gives Netezza another point of optimization: because the
original compressed data blocks are still in memory, they can be
automatically reused in later queries requiring similar data
through the table cache—an automated mechanism requiring no
administrator involvement.
Platform for Advanced Analytics
Traditionally, analytics had to be built and deployed on separate analytics
servers. These servers would run computationally intensive analytical algo-
rithms and interface with a data repository, such as a data warehouse, on the
back end. This architecture lengthens the time from model inception to deploy-
ment, and requires data movement from the data repository to the analytics
server. Not only does this process take too much time, but it's inefficient,
limits the data that can be used to derive insight, constrains the scope of the
analytic modeling, and impedes the ability to experiment iteratively.
Netezza offers a distinctive and simple-to-use approach for serious ana-
lytics on structured data with IBM Netezza Analytics, an advanced
