Databases Reference
In-Depth Information
FIGURE 7.2
Data warehouse platform.
Platform engineering
With advances in technology, there are several choices to enable platform engineering. This is funda-
mentally different from replatforming, where you can move the entire data warehouse. With a plat-
form engineering approach, you can modify pieces and parts of the infrastructure and get great gains
in scalability and performance.
The concept of platform engineering was prominent in the automotive industry where the focus
was on improving quality, reducing costs, and delivering services and products to end users in a
highly cost-efficient manner. By following these principles, the Japanese and Korean automakers
have crafted a strategy to offer products at very competitive prices while managing the overall user
experience and adhering to quality that meets performance expectations. Borrowing on the same prin-
ciples, the underlying goal of platform engineering applied to the data warehouse can translate to:
●
Reduce the cost of the data warehouse.
●
Increase efficiencies of processing.
●
Simplify the complexities in the acquisition, processing, and delivery of data.
●
Reduce redundancies.
●
Minimize customization.
●
Isolate complexity into manageable modular environments.
There are several approaches to platform engineering based on the performance and scalability
needs of the data warehouse; the typical layers of a platform include those shown in
Figure 7.2
.
Platform reengineering can be done at multiple layers:
●
Storage level.
In this approach the storage layer of the data is engineered to process data at very high
speeds for high or low volumes. This is not an isolation exercise. When storage is customized, often
the network and the operating system are also modified to process data at twice the speed of the
storage to manage multiple cycles of data transfer between the storage and the database servers. This
is not a very popular option, as it needs large-scale efforts, the ROI question cannot be answered in
simple math, and the total operating costs often surpass the baseline expectations.
●
Server reengineering
. In this approach the hardware and its components can be replaced with more
modern components that can be supported in the configuration. For example, replacing processors,
memory upgrades, and network card upgrades are all common examples of server platform
reengineering. The issue with server reengineering is the scalability is limited by the processer
design and flexibility and by the underlying operating system design that can leverage the processor
capabilities. Due to the dual impact, the scalability from server upgrades is very limited. Though
