Information Technology Reference
In-Depth Information
Metadata
Most data in a company's database is not scientific in the sense that it
does not come from scientific observations or observations of scientific
experiments. It may be objective and accurate, but it is driven by agreed-
upon assertions or definitions. For example, the “duration of a call to a
call center” may be picked up very accurately from the system clocks but
the
of start and end times, of a call is set by the designers. The
knowledge that one draws from such data — through business intelligence
or regular reports — lacks scientific rigor. Trying to compare the length
of support calls across companies when they are operating to different
definitions of duration is meaningless.
Right data can go wrong. Data must be understood in its context,
which may or may not be the metadata in its conventional definition. This
often happens with respect to codes. If one sees a record that says
definition
Joe
belonged to “Dept 17” in 1982, then unless one has access to the Dept
code table as of 1982, one cannot assume that the “Dept 17” today is the
same as that in 1982. In
, this is the issue of slowly
changing dimensions. Not recognizing and recording this change can
render old data, which was accurate or right when recorded, useless or
ambiguous at a later date.
When a data entry form changes, it could be driven by a change in the
process or some other external factor. For example, a new legal requirement
may be driving the need to get additional information. Such changes mean
that the data in a database could have come through old and new versions
of forms. If fields before a certain form version are blank, there could be
a reason for it. Tracking the history of form changes is important. There
may be metadata about the information reflecting this history. Most of this
information is not captured properly or is difficult to determine.
data warehousing
Certainty of Data
Data does not have Platonic certainty. The data we store may be right or
wrong. It does not become right just because it is recorded in a formal
system. An employee may give his or her date of birth to a company and
it could be recorded with or without verification. Unless this information
(verified or not) was captured, one would not be able to say that the
employee's date of birth was, in fact, correct. It is not sufficient to turn
to documents defining verification processes because, while it is true that
such a process existed at the time of capture of the information, there is
no certainty as to whether the process was followed for this particular
employee.
