Data Conversion: Doing it Right the First Time - High-Performance Web Databases: Design, Development, and Deployment - page 597

fields, obscure coding schemes, or undocumented relationships. If the doc-
umentation is thorough, many data conversion pitfalls can be avoided.
Possible Mapping Patterns
The mapping of old to new is usually very complex. There seems to be
no useful canonical scheme for dealing with this set of problems. Each new
conversion seems to consist of myriad special cases. In the general case, a
given new field may depend on the values found in multiple fields con-
tained in multiple records of a number of files. This works the other way as
well — one field in an old record may be assigned to different fields or even
to different tables, depending on the values encountered.
If the conversion also requires intelligent handling of updates and de-
letes to the old system, the problem is complicated even further. This is
true when one source file is split into several destination files and, at the
same time, one destination file receives data from several source files.
Then, if just one record is deleted in a source file, some fields will have to
be set to null in the destination file, but only those coming from the deleted
source record. This method, however, may violate some of the integrity
rules in the new database.
It may be best to specify the mapping in simple tabular and textual fash-
ion. Each new field will have the corresponding old fields listed, along with
any special translation rules required. These rules could be documented as
decision tables, decision trees, pseudo code, or action diagrams.
Relational Mathematics
In database theory, it is possible to join together all fields in a database
in a systematic manner and to create what is called the “universal relation.”
Although this technique has little merit as a scheme for designing or imple-
menting a database, it may be a useful device for thinking about the map-
ping of old to new. It should be possible to specify any complex mapping as
a view based on the universal relation. The relational algebra or the rela-
tional calculus could be used as the specification medium for detailing the
rules of the mapping in a declarative fashion.
DESIGNING THE CONVERSION
Before starting to design a computer program, reentering the data man-
ually from source records should be considered as a possibility.
Special Requirements for Data Warehousing
Data warehousing assumes that the conversion issue arises on a routine,
periodic basis. All of the problems that arise in a one-time conversion must