Multiple Comparison Procedures - Analysis of Variance Designs - page 113

In defense of the a posteriori strategy, it is very often the case that
in behavioral and social science research we may not have sufficiently
developed theories to test them as specifically as is called for by using
planned comparisons. Related to this defense, much of behavioral and
social science research is really exploratory. Therefore, examining many
or all of the mean differences to find out what happened in the study
may help researchers to develop models of the phenomenon under study,
which can then be more rigorously tested in subsequent research studies.
On the other hand, it is not justifiable to use a posteriori multiple
comparisons for reasons having nothing to do with the nature of the
research; in fact, doing so runs a risk of compromising the integrity of
the study. We believe that there are two inappropriate and not necessarily
mutually exclusive reasons driving some researchers to use unplanned
comparisons: (a) the procedures are so incredibly convenient to use in
SPSS and SAS that researchers automatically invoke them under all design
conditions whether they are appropriate or not, and (b) researchers some-
times do not know or consider that alternative procedures, perhaps more
appropriate to their experimental design and certainly almost as con-
venient to use as unplanned comparisons, are available to them. We
do not endorse using a posteriori comparisons under these two sets of
conditions.
7.3 PAIRWISE VERSUS COMPOSITE COMPARISONS
We have been talking about mean differences in Chapter 6 and early in
this chapter as though we always compare the mean of a single condition
to the mean of some other single condition. That sort of comparison is
known as a pairwise or simple comparison and, to be sure, is frequently
performed. However, other possibilities exist.
It is also possible to compare two means where one or both of them
are weighted linear composites made up of the means of two or more
conditions. These sorts of comparisons are known as nonpairwise, com-
posite, or complex comparisons. We will be more explicit about the idea
of a weighted linear composite later in the chapter when we discuss user-
defined contrasts; to keep it simple for now, just think of the compos-
ite as an average of two or more means with the means being equally
weighted.
We can use our SAT study time example from the last chapter to illus-
trate a couple of different composite comparisons. Recall that separate
groups prepared either zero, two, four, six, or eight months for the test. In
the type of nonpairwise comparison known as one-to-composite compari-
son, we could ask if any study time made a difference over not preparing
at all. The comparison here would take the following form:
Group 0 mean versus (combined mean of Groups 2, 4, 6, 8).