Databases Reference
In-Depth Information
Client Program (BI or SQL
Application)
Tr anslation Layer
(Semantic)
SQL/MapRedece Libraries
SQL & MapRedece
Interface
Data
Data
Data
Data
Data
Data
Data
Data
FIGURE 4.13
Conceptual SQL/MapReduce architecture.
SQL/MapReduce
Business intelligence has been one of the most successful applications in the last decade, but severe
performance limitations have been a bottleneck, especially with detail data analysis. The problem
becomes compounded with analytics and the need for 360° perspective on customers and products
with ad-hoc analysis demands from users. The powerful combination of SQL when extended to
MapReduce enables users to explore larger volumes of raw data through normal SQL functions and
regular business intelligence tools. This is the fundamental concept behind SQL/MapReduce. There
are a few popular implementations of SQL/MapReduce including Hive, AsterData, Greenplum, and
HadoopDB.
Figure 4.13
shows a conceptual architecture of a SQL/MapReduce implementation. There are a
few important components to understand:
●
Translator—this is a custom layer provided by the solution. It can simply be a library of functions
to extend in the current database environment.
●
SQL/MapReduce interface—this is the layer that will create and distribute the jobs at the lowest
MapReduce execution layer.
●
SQL/MapReduce libraries—catalog of library functions.
The overall benefits of combining SQL/MapReduce include:
●
Use of SQL for powerful postresult analytics and MapReduce to perform large-scale data
processing on unstructured and semi-structured data.
●
Effectively use the sharding capabilities of MapReduce to scale up and scale out the data
irrespective of volume or variety.
●
Provide the business user all the data with the same interface tool that runs on SQL.
