Database Reference
In-Depth Information
Failure toconsider these points will result in a warehouse that either doesn't
fully leverage the new hardware or is heavily fragmented and therefore
incapable of issuing large block I/O down to the new storage subsystem.
Thiswouldseriouslyimpactperformance,slowingdownresponsivenessand
ultimately impacting the availability of the data warehouse.
Fragmentation
When loading data into a data warehouse, you need to be mindful of
fragmentation. Unlike transactional systems (online transactional
processing [OLTP]), data warehouses observe a scan-centric I/O pattern
that demands largely fragmentation-free tables.
As the size of the tables increase, this issue becomes more apparent and
significantly more important. At a small scale, many warehouses benefit
purely from moving from legacy, often virtualized, environments to
dedicated platforms. The performance bump observed masks the
underlying issues of fragmentation in database tables.
However, as the system grows, that performance bump is insufficient.
Fragmentation needs to be addressed. This can often lead to a significant
rewrite of data warehouse extract, transform, and load (ETL) software. This
is no small undertaking. It is often said that the ETL of a data warehouse
accounts for 75% of the effort in a data warehouse project. Consequently,
these rewrites can attract a significant and often hidden cost. All the
business sees is that changes take longer to deliver as service levels suffer.
Return on Investment
When dealing with a traditional SMP warehouse architecture, we have to
consider how the costs stack up. Many of the costs when scaling up are
clearly understood. If I buy a bigger box with more cores, it will cost me
more money. The problem is that, say, doubling the cores doesn't simply
double the cost. Often, it is more than double. It's a question of economies
of scale, supply and demand, and of course, the fact that building bigger
integrated boxes is a more complex and more valuable offering. This
translates to nonlinear pricing, which moves further and further away from
the line as the offering becomes less of a commodity.
And that's just the technical questions. What about the human resource
capital employed to manage this system? As previously mentioned, a bigger,
more sophisticated system requires greater levels of technical support and
