Data Conversion: Doing it Right the First Time - High-Performance Web Databases: Design, Development, and Deployment - page 595

deleted, the employee record will be orphaned, and if an employee changes
departments, the employee key will have to change. When doing a conver-
sion, it would be desirable to remove the intelligent key structure.
Other Problems
Often other problems with the old data cannot be classified easily. These
problems involve errors in the data that cannot be detected except by go-
ing back to the source, or violations of various arcane constraints that have
not been programmed as edit checks in the existing system. There may be
special rules that tie field values to multiple records, multiple fields, or mul-
tiple files. Although they may not have a practical implication for the con-
version effort, if these problems become obvious, they might falsely be
attributed to the conversion routines.
THE ERROR CORRECTION PROCESS
The data correction effort should be run as part of a separate sub-
project. The DCT should determine whether the resources to correct the
data can be made available. A wholesale commitment from the owners of
the data will be required, and probably a commitment of programming re-
sources as well. Error correction cannot be done within the context of rap-
id applications development (RAD).
Resources for the Correction Effort
Concerning resources for the correction effort, the best-case scenario
would ensure that:
• Resources are obtained from the client if a major correction effort is
required.
• Management pays adequate attention to the issue if a data-quality
problem is identified.
• The sources of the problem will be identified in a fair and nonjudgmen-
tal manner if a data-quality problem is identified.
Choices for Correction
The effort required to write an edit program to look for errors is consid-
erable, and chances are good that this will be part of the conversion code
and not an independent set of audit programs. Some of the errors may be
detected before conversion begins, but it is likely that many of the prob-
lems will be found during the conversion run.
Once data errors are discovered, data can be copied as is, corrected, or
abandoned. The conversion programs should reject erroneous transactions
and provide reports that explain why data was rejected. If the decision is