Database Reference
In-Depth Information
•
Pattern 3
,
Extract, Transform, Load, and Transform
(
ETLT
): This is a
combination of the previous two alternatives, where we choose to leverage
all the in-built transformation and scrubbing functions of Informatica and go
tothetarget forallcomplextransformations that might/mightnotinvolvelarge
volumes.
Most of the data integration tools in the market (including Informatica PowerCenter)
support a feature called
Pushdown Optimization
(
PDO
). The Pushdown Optimiza-
tion technique helps achieve optimal performance by load balancing the processing
across the servers. Let us take an example of transformation logic that requires filter-
ing of data based on a condition that requires to lookup data from the database table
with a large number of rows. Instead of loading data onto the data integration tier
and processing the filter logic, running an SQL query on the database could prove
to be optimal. This is the case where the transformation logic is pushed down to be
executed at the target database level rather than at the source. This is the ELT case.
Before we examine the suitable data loading pattern, we would need to consider the
following points:
• Identify if data load throughput is a crucial requirement
• Estimate the current workload on the source and target platforms
• Measure the cost of hardware and software to add additional computing re-
sources to the target environment
• Guesstimate relative efficiency of performing a particular operation in the
source, target, or integration system
The following table provides a comparative analysis of the proposed data loading
patterns:
Search WWH ::
Custom Search