Information Technology Reference
In-Depth Information
CHAPTER SEVEN
Multiple Comparison Procedures
7.1 OVERVIEW
Once we have determined that an independent variable has yielded a sig-
nificant effect, we must next turn our attention to the differences between
the means of the conditions in the study. If there are only two means, then
we automatically know that they are significantly different. With three or
more means a significant
F
ratio reveals only that there is a difference
between at least one pair of means in the design; in this case we must
perform an additional, post-ANOVA
multiple comparison procedure
to
determine which of the three or more means differ from which others.
There are a variety of multiple comparison procedures that are avail-
able to researchers. Before presenting them, we will first describe some
dimensions along which they differ; this will help us, as we go through
this chapter, to discuss the differences among them.
7.2 PLANNED VERSUS UNPLANNED COMPARISONS
In an idealized world of research, hypotheses regarding differences bet-
ween certain of the groups in a design are formulated in advance of the
data collection on the basis of the theoretical context out of which the
research was generated. Once the study is completed and a statistically
significant
F
ratio is obtained in the omnibus ANOVA (although in this
idealized world a statistically significant
F
ratiomaynotevenbenecessary
if there are a very few hypothesized mean differences), the researchers
then carry out the mean comparisons that they have already specified.
Comparisons made under this approach are labeled as
planned
or
a priori
comparisons (Kirk, 1995).
In the practical world of research, most of the time multiple com-
parisons are performed on an
unplanned
or
a posteriori
basis (Everitt,
2001) without the researchers having formulated specific hypotheses at
the outset and/or without restricting themselves to examining only the
hypothesized mean differences. Under this strategy, if and only if a statis-
tically significant
F
ratio is obtained in the omnibus ANOVA, researchers
examine a variety of aspects of the results - including all possible pairwise
combinations of the group means - to determine which of them reveal
statistically significant mean differences.
112
