THE DATA WAREHOUSE - Fundamentals of Database Management Systems - page 361

Data Expertise
• An understanding of the company's transactional data and databases for
selection and integration into the data warehouse.
• An understanding of the company's transactional data and databases to design
and manage data cleaning and data transformation as necessary.
• Familiarity with outside data sources for the acquisition of enrichment data.

■

Technical Expertise
• An understanding of data warehouse design principles for the initial design.
• An understanding of OLAP and data mining techniques so that the data
warehouse design will properly support these processes.
• An understanding of the company's transactional databases in order to manage
or coordinate the regularly scheduled appending of new data to the data
warehouse.
• An understanding of handling very large databases in general (as the data
warehouse will inevitably be) with their unique requirements for security,
backup and recovery, being split across multiple disk devices, and so forth.

■

The other issue in administering a data warehouse is metadata; i.e., the data
warehouse must have a data dictionary to go along with it. The data warehouse is
a huge data resource for the company and has great potential to give the company
a competitive advantage. But, for this to happen, the company's employees have to
understand what data is in it! And for two reasons. One is to think about how to use
the data to the company's advantage, through OLAP and data mining. The other is
actually to access the data for processing with those techniques.

CHALLENGES IN DATA WAREHOUSING

Data warehousing presents a distinct set of challenges. Many companies have
jumped into data warehousing with both feet, only to find that they had bitten
off more than they could chew and had to back off. Often, they try again with
a more gradual approach and eventually succeed. Many of the pitfalls of data
warehousing have already been mentioned at one point or another in this chapter.
These include the technical challenges of data cleaning and finding more ''dirty''
data than expected, problems associated with coordinating the regular appending
of new data from the transactional databases to the data warehouse, and difficulties
in managing very large databases, which, as we have said, the data warehouse will
inevitably be. There is also the separate challenge of building and maintaining the
data dictionary and making sure that everyone who needs it understands what's in
it and has access to it.
Another major challenge of a different kind is trying to satisfy the user
community. In concept, the idea is to build such a broad, general data warehouse
that it will satisfy all user demands. In practice, decisions have to be made about
what and how much data it is practical to incorporate in the data warehouse at a given
time and at a given point in the development of the data warehouse. Unfortunately,
it is almost inevitable that some users will not be satisfied in general with the data
at their disposal and others will want the data warehouse data to be modified in
some way to produce better or different results. And that's not a bad thing! It means
that people in the company understand or are gaining an appreciation for the great