Testing Hypotheses: Choosing a Test Statistic - Common Errors in Statistics - page 74

to determine the appropriate size of resistor to use in an electric circuit, to
determine the date of the next eclipse of the moon, and to establish the
identity of the criminal (perhaps from the fact the dog did not bark on the
night of the crime). Find the formula, plug in the values, turn the crank,
and out pops the result (or it does for Sherlock Holmes, 11 at least).
When we assert that for a given population that a percentage of samples
will have a specific composition, this is a deduction also. But when we
make an inductive generalization about a population based upon our
analysis of a sample, we are on shakier ground. Newton's Law of gravi-
tation provided an exact fit to observed astronomical data for several
centuries; consequently, there was general agreement that Newton's gener-
alization from observation was an accurate description of the real world.
Later, as improvements in astronomical measuring instruments extended
the range of the observable universe, scientists realized that Newton's
Law was only a generalization and not a property of the universe at all.
Einstein's Theory of Relativity gives a much closer fit to the data, a fit that
has not been contradicted by any observations in the century since its for-
mulation. But this still does not mean that relativity provides us with a
complete, correct, and comprehensive view of the universe.
In our research efforts, the only statements we can make with God-like
certainty are of the form “our conclusions fit the data.” The true nature of
the real world is unknowable. We can speculate, but never conclude.
The gap between the sample and the population will always require a
leap of faith. We understand only in so far as we are capable of under-
standing [Lonergan, 1992].

SUMMARY
Know your objectives in testing. Know your data's origins. Know the
assumptions you feel comfortable with. Never assign probabilities to the
true state of nature, but only to the validity of your own predictions. Col-
lecting more and better data may be your best alternative.

TO LEARN MORE
For commentary on the use of wrong or inappropriate statistical methods,
see Avram et al. [1985], Badrick and Flatman [1999], Berger et al.
[2002], Bland and Altman [1995], Cherry [1998], Dar, Serlin, and Omer
[1997], Elwood [1998], Felson, Cupples, and Meenan [1984], Fienberg
[1990], Gore, Jones, and Rytter [1977], Lieberson [1985], MacArthur

11

See “Silver Blaze” by A. Conan-Doyle, Strand Magazine , December 1892.