What’s Wrong With My Data? - High-Performance Web Databases: Design, Development, and Deployment - page 411

Chapter 27

What's Wrong
With My Data?

Jeffery Feldman

Y , “G ,

OU HAVE PROBABLY HEARD THE EXPRESSION ARBAGE IN GARBAGE

.”

OUT A computer system utilizing the most technologically sophisticated

architecture, elegant user interfaces, and intuitive report generation capa-
bilities, can be rendered nearly useless if the data it relies upon is funda-
mentally flawed. While investigating and correcting “bad” data may be
among the least glamourous tasks to perform, it is absolutely critical if or-
ganizations are to realize the full potential of the systems they have devel-
oped (often at a considerable cost in time, resources, and dollars).

This paper illustrates a variety of potential quality problems that could
be present within the files and databases of any organization. These prob-
lems will be especially evident in older legacy systems, although not nec-
essarily restricted to older systems by any means. Knowing the possible
issues is the first step towards addressing them. Further steps would
include taking action to correct the data, and (perhaps even more impor-
tantly) taking action to correct the processes that led to flawed data being
stored in the first place. This paper focuses on describing those possible
flaws.

WHY SHOULD I CARE?

Perhaps you are not quite convinced that data integrity should be a
major concern, or at any rate not for the applications you are dealing with.
After all, the existing systems are running just fine with the current data,
aren't they? And any required data cleansing can be done at some point in
the future, when time and resources become more available.

Well, maybe. Much depends on just how the data is being used, and on
the organization's tolerance for risk. Let us say, for example, that the only
use of the data is to support a company-internal management information
system, and that about 5 percent of the data is known to be in error. A cost-
benefit analysis that compares the effort required to correct the data vs.