Database Reference
In-Depth Information
multiple steps or components. A good place to start is learning more about
the origin of the data and the process for collecting it.
When formal statistical evidence of reliability is not presented, we are left to
use our own judgment and intuition about the reliability of the data and the
process for gathering it.
There are also a number of ways to determine the data's validity. The most
important and easiest of these is to ask whether the data has
face validity
. Do
the numbers make sense? Are they consistent with past trends? If I weigh myself
each day on the same reliable scale and it says 185, and then I weigh myself
on a different scale and it says 225, does this mean I've suddenly gained 40
pounds? Of course not. Because the first scale is reliable, I know that the new
scale is not valid. In education, if a student that consistently scores high on
standardized exams designed to measure intelligence and academic ability
fails the final test, I would think that the last test is not a valid measure of the
student's ability. Perhaps she was sick that day or had just found out that her
parents were getting a divorce.
Another consideration is to think about the data in relation to other measures.
And although these relationships can be formally tested with statistics, it is
important to do quick, informal checks for the validity of relationships between
the data that you might expect to see. For instance, does an increase in the
number of hours logged by a sales team also correspond to an increase in
the number of sales? Does the retirement of a number of teachers in a school
(without their replacement) lead to a higher student-teacher ratio? Does the
multimillion dollar advertising campaign spotlighting a revamped website lead
to increased search and visits to that site? Often these predictable relation-
ships can help establish validity.
Finally, one must consider the selection process in determining the validity
of data. Again, a number of questions are helpful:
Who does the data represent?
⬛
Is it a sample or does it include everyone?
⬛
If it is a sample, how were they chosen? Why? Are the numbers “cherry-
picked” to tell a specific story or sell a certain idea? Or was it random?
⬛
Is the sample representative of the target population of interest? That is,
if it is a sample of U.S. hospitals, for example, were only hospitals in major
urban cities on the east coast chosen, or was the information drawn from
a range of urban and rural hospitals from across the country?
⬛
Search WWH ::
Custom Search