Database Reference
In-Depth Information
• It supports linear scalability, also referred to as elastic scalability, by adding
a new node to the cluster, additional storage, and processing capability, both
in terms of load performance and query performance is gained
Data loading patterns
Fromwhatwehavelearnedaboutdatawarehousing,itisveryevidentthatdataload-
ing forms a major process. This process is responsible for pulling data from various
source systems and consolidating it into a warehouse.
The data loading function is beyond just extracting and loading data. It involves data
scrubbing, transformation, and cleansing processes that should be driven using con-
figurable business rules and requires a standard data definition/metadata in place.
In this section, we will explore various data loading patterns that can be considered
for implementing complex transformations (transformations that are done on higher
volumes of data and require frequent lookups of data references from the underlying
database).
There are three alternatives that can be considered for deciding on where the trans-
formation or data scrubbing logic should reside.
Pattern 1 , Extract, Transform, and Load ( ETL ): This is the case where the
transformations are done within the data integration tier and the final data is
pushed onto the target (database).
Pattern 2 , Extract, Load, and Transform ( ELT ): In this case the data is
loaded in an efficient manner onto the target (database) and the entire trans-
formation is done at the target.
Search WWH ::




Custom Search