Data Conversion: Doing it Right the First Time - High-Performance Web Databases: Design, Development, and Deployment - page 589

from the old. In this case, the dummy value strategy may be appropriate,
but it must be explained fully to the client.
Identifying Problems
Data problems only can be detected after the old data structure is fully
understood. Once it is determined what the new model will look like, a
deeper analysis of the issue can be done.
A full analysis of the issue includes looking for erroneous information,
missing information, redundancies, inconsistencies, missing keys, and any
other problem that will make the conversion difficult or impossible without
a lot of manual intervention. Any findings should be documented and
brought to the attention of the client. Information must be documented in
a fashion that makes sense to the client.
Once the problems have been identified, the DCT can help the client
identify a corrective strategy. The client must understand why errors have
been creeping into the systems. The cause is usually a mixture of problems
with the old data structure, problems with the existing input system, and
data entry problems that have been ongoing. It may be that the existing
system does not reflect the business properly. The users may have been
working around the system's deficiencies for years in ways that violated its
integrity. In any case, the new system should be tighter than the old one at
the programming and database level, should reflect the business properly,
and the new procedures should not result in problems with usability or
data quality.
Documenting the Requirements
After the initial study of the conversion is done, the findings should be
documented. Some of this work will have been done as part of the regular
system design. There must also be a design for the conversion programs,
whether it is a one-time or an ongoing activity. First-time as well as ongoing
load requirements must be examined.
Estimates should include the time necessary to extract, edit, correct,
and upload data. Costs for disk storage and CPUs also should be projected.
In addition, the sizing requirements should be estimated well in advance of
hardware purchases.
Correcting the Data
The client may want to correct the data before the conversion effort be-
gins or may be willing to convert the data over time. It is best to make sure
that the data that is converted is error-free, at least with respect to the for-
mal integrity constraints defined for the new model.