Database Reference
In-Depth Information
Challenges Faced by Traditional Data Warehouse
Architectures
What is a traditional architecture when it is at home? Every warehouse
I come across certainly has some distinguishing features that make it
different or special. However, many of these warehouses usually share one
very common element in their architecture: they are designed to a
symmetric multiprocessor (SMP) architecture; that is, they are designed to
scale
up
. When you exceed the capabilities of the existing server, you go out
and buy a bigger box. This is a very effective pattern for smaller systems, but
it does have its drawbacks. These drawbacks become particularly evident
when we start to consider some of the challenges businesses face today.
Later in this section we will cover scale-out approaches, but first we will
focus on the inherent issues with scale-up.
Technical Constraints
This section identifies, qualifies, and discusses some of the technical
challenges facing data warehouses in today's environments. You will
recognize many of these challenges from your own environments. This list is
not intended to be exhaustive, but it does cover a number of the issues:
• Scaling compute
• Shared resources
• Data volumes and I/O throughput
• Application architecture changes
• Return on investment
Scaling Compute
Scaling compute is hard, especially if you have designed an SMP
architecture. You are limited on two fronts: server and software.
Server
The first question to ask yourself is this: How many CPU sockets do you
have at your disposal? 1, 2, 4, or 8? Next, what CPUs are you using to fill
them? 4, 6, 8, 10 core? At best you could get up to 80 physical cores. With
hyper-threading, you could double this figure to get to 160 virtual CPUs.
