Collecting Data - Common Errors in Statistics - page 33

If the mother is the one treated with the drug, then the mother is the
experimental unit, not the fetus. A litter of six or seven corresponds only
to a sample of size one.
As for the topical ointment, while more precise results might be
obtained by treating only one eye with the new ointment and recording
the subsequent difference in appearance between the treated and untreated
eyes, each patient still yields only one observation, not two.
Identically Distributed Observations
If you change measuring instruments during a study or change observers,
then you will have introduced an additional source of variation and the
resulting observations will not be identically distributed.
The same problems will arise if you discover during the course of a
study (as is often the case) that a precise measuring instrument is no
longer calibrated and readings have drifted. To forestall this, any measur-
ing instrument should have been exposed to an extensive burn-in before
the start of a set of experiments and should be recalibrated as frequently as
the results of the burn-in or pre-study period dictate.
Similarly, one doesn't just mail out several thousand copies of a survey
before performing an initial pilot study to weed out or correct ambiguous
and misleading questions.
The following groups are unlikely to yield identically distributed obser-
vations: the first to respond to a survey, those who only respond after
been offered an inducement, and nonresponders.
EXPERIMENTAL DESIGN
Statisticians have found three ways for coping with individual-to-individual
and observer-to-observer variation:
1. Controlling. Making the environment for the study—the subjects,
the manner in which the treatment is administered, the manner in
which the observations are obtained, the apparatus used to make
the measurements, and the criteria for interpretation—as uniform
and homogeneous as possible.
2. Blocking. A clinician might stratify the population into subgroups
based on such factors as age, sex, race, and the severity of the con-
dition and restricting comparisons to individuals who belong to
the same subgroup. An agronomist would want to stratify on the
basis of soil composition and environment.
3. Randomizing. Randomly assigning patients to treatment within
each subgroup so that the innumerable factors that can neither be
controlled nor observed directly are as likely to influence the
outcome of one treatment as another.