Information Technology Reference
In-Depth Information
But the study designers hadn't considered these issues. A precise and
costly survey was now a matter of guesswork.
You can always group your results (and modify your groupings) after a
study is completed. If after-the-fact grouping is a possibility, your design
should state how the grouping will be determined; otherwise there will be
the suspicion that you chose the grouping to obtain desired results.
Experiments
Measuring devices differ widely both in what they measure and in the pre-
cision with which they measure it. As noted in the next section of this
chapter, the greater the precision with which measurements are made, the
smaller the sample size required to reduce both Type I and Type II errors
below specific levels.
Before you rush out and purchase the most expensive and precise mea-
suring instruments on the market, consider that the total cost
C
of an
experimental procedure is
S
+
nc
, where
n
is the sample size and
c
is the
cost per unit sampled.
The startup cost
S
includes the cost of the measuring device.
c
is made
up of the cost of supplies and personnel costs. The latter includes not only
the time spent on individual measurements but also the time spent in
preparing and calibrating the instrument for use.
Less obvious factors in the selection of a measuring instrument include
impact on the subject, reliability (personnel costs continue even when an
instrument is down), and reusability in future trials. For example, one of
the advantages of the latest technology for blood analysis is that less blood
needs to be drawn from patients. Less blood means happier subjects, fewer
withdrawals, and a smaller initial sample size.
Surveys
While no scientist would dream of performing an experiment without first
mastering all the techniques involved, an amazing number will blunder
into the execution of large-scale and costly surveys without a preliminary
study of all the collateral issues a survey entails.
We know of one institute that mailed out some 20,000 questionnaires
(didn't the post office just raise its rates again?) before discovering that
half the addresses were in error and that the vast majority of the remain-
der were being discarded unopened before prospective participants had
even read the “sales pitch.”
Fortunately, there are texts such as Bly [1990, 1996] that will tell you
how to word a “sales pitch” and the optimal colors and graphics to use
along with the wording. They will tell you what “hooks” to use on the
envelope to ensure attention to the contents and what premiums to offer
to increase participation.
