Databases Reference
In-Depth Information
FIGURE 10.7
Integration-driven approach.
●
Pitfalls to avoid:
●
Too much data complexity at any one layer of processing.
●
Poor metadata.
●
Incorrect data analysis within Big Data layers.
●
Incorrect levels of integration (at data granularity) within the Big Data layers.
●
Incorrect application of data bus integration.
Hadoop & RDBMS
Figure 10.7
shows the integration-driven approach to creating the next-generation data warehouse.
To create the next-generation data warehouse we combine the Big Data processing platform created
in Hadoop or NoSQL and the existing RDBMS-based data warehouse infrastructure by deploying a
connector between the two systems. This connecter will be a bridge to exchange data between the
two platforms. At the time of writing, most of the RDBMS, BI, analytics, and NoSQL vendors have
developed Hadoop and NoSQL connectors.
Workload processing is this architecture blends data processing across both platforms, providing
scalability and reducing complexity. The streamlining of workload creates a scalable platform across
both the infrastructure layers, where data discovery can be seamlessly enabled in either platform. The
complexity of this architecture is the dependency on the performance of the connector. The connec-
tors largely mimic a JDBC behavior, and the bandwidth to transport data will be a severe bottleneck,
considering the query complexity in the data discovery process.
