Collecting Data - Common Errors in Statistics - page 32

telephone and responded to the survey favored Landon; those who voted
did not. An economic study may be flawed because we have overlooked
the homeless, 2 and an astrophysical study may be flawed because of over-
looking galaxies whose central surface brightness was very low. 3

FUNDAMENTAL ASSUMPTIONS
Most statistical procedures rely on two fundamental assumptions: that the
observations are independent of one another and that they are identically
distributed. If your methods of collection fail to honor these assumptions,
then your analysis must fail also.

Independent Observations
To ensure the independence of responses in a return-by-mail or return-by-
web survey, no more than one form per household should be accepted. If
a comparison of the responses within a household is desired, then the
members of the household should be interviewed separately, outside of
each other's hearing, and with no opportunity to discuss the survey in
between. People care what other people think and when asked about an
emotionally charged topic may or may not tell the truth. In fact, they are
unlikely to tell the truth if they feel that others may overhear or somehow
learn of their responses.
To ensure independence of the observations in an experiment, deter-
mine in advance what constitutes the experimental unit .
In the majority of cases, the unit is obvious: One planet means one
position in space, one container of gas means one volume and pressure to
be recorded, and one runner on one fixed race course means one elapsed
time.
In a clinical trial, each individual patient corresponds to a single set of
observations or does she? Suppose we are testing the effects of a topical
ointment on pinkeye. Is each eye a separate experimental unit, or each
patient?
It is common in toxicology to examine a large number of slides. But
regardless of how many are examined in the search for mutagenic and
toxic effects, if all slides come from a single treated animal, then the total
size of the sample is one.
We may be concerned with the possible effects a new drug might have
on a pregnant woman and, as critically, on her children. In our prelimi-
nary tests, we'll be working with mice. Is each fetus in the litter a separate
experimental unit, or each mother?

2

City of New York v. Dept of Commerce, 822 F. Supp. 906 (E.D.N.Y., 1993).

3

Bothun [1998, p. 249].