Information Technology Reference
In-Depth Information
[1999], Jones and Tukey [2000], McBride, Loftis, and Adkins [1993],
Nester [1996], Parkhurst [2001], and Suter [1996].
The vast majority of such cautions are unnecessary provided that we
treat
p
values as merely one part of the evidence to be used in decision-
making. They need to be viewed and interpreted in the light of all the sur-
rounding evidence, past and present. No computer should be allowed to
make decisions for you.
A failure to reject may result from insensitive or inappropriate measure-
ments, or too small a sample size.
A difference that is statistically significant may be of no practical inter-
est. Take a large enough sample and we will always reject the null hypoth-
esis; take too small a sample and we will always accept—to say nothing of
“significant” results that arise solely because their authors chose to test a
“null” hypothesis rather than one of practical interest. (See Chapter 4.)
Many researchers would argue that there are always three regions to
which a statistic may be assigned: acceptance, rejection, and indifference.
When a statistic falls in the latter, intermediate region it may suggest a
need for additional experiments. The
p
value is only one brick in the wall;
all our other knowledge must and should be taken into consideration
(Horwitz et al., 1998).
SUMMARY
•
Provide details of power and sample size calculations.
•
Describe treatment allocation.
•
Detail exceptions including withdrawals and other sources of
missing data.
•
Use meaningful measures of dispersion.
•
Use confidence intervals in preference to
p
values.
•
Report sources of bias.
•
Formal statistical inference is appropriate only for randomized
studies and predetermined hypotheses.
TO LEARN MORE
The text by Lang and Secic [1997] is must reading; reporting criteria for
meta-analyses are given on page 177 ff. See Tufte [1983] on the issue of
table versus graph. For more on the geometric versus arithmetic mean see
Parkhurst [1998]. For more on reporting requirements, see Begg et al.
[1996], Bailar and Mosteller [1988], Grant [1989], Altman et al. [2001;
the revised CONSORT statement], and International Committee of
Medical Journal Editors [1997].
