Multiple Comparison Procedures - Analysis of Variance Designs - page 114

As you can see, we would address the question of whether any study
time made a difference by comparing the group of students who did no
preparation at all to the rest of the groups combined. For this set of com-
bined groups, we would probably just compute a simple overall mean for
Groups 2, 4, 6, and 8 (each group would be weighted equally) and com-
pare this linear composite to the mean of the students who did not study
at all.
In the type of nonpairwise comparison known as composite-to-
composite comparison, we could ask if a good deal of preparation resulted
in higher test scores than some modest amount of preparation. Such a
comparison could take this form:

(combined mean of Groups 2 and 4) versus (combined
mean of Groups 6 and 8).

Here, we could combine the groups having studied two and four months,
probably weighting them equally, and compare them to a combination of
the groups having studied six or eight months, probably weighting them
equally as well.
There are at least two major reasons for considering composite com-
parisons as viable alternatives when you are thinking about your multiple
comparisons. First, these may help you better understand the pheno-
menon under study. That is, even though you took the trouble to dis-
tinguish between certain conditions in the original design of the study, a
more relevant hypothesis may concern combining certain conditions in
the post-ANOVA phase of data analysis. Second, this may be at least in part
what the significant F ratio was detecting. We keep saying that a statisti-
cally significant F ratio tells you that there is a significant mean difference
someplace. Well, the most interesting mean differences are sometimes
based on a nonpairwise comparison.

7.4 ORTHOGONAL VERSUS NONORTHOGONAL COMPARISONS

Multiple comparisons may be made on either an orthogonal or non-
orthogonal basis. Comparisons that are orthogonal are mutually indepen-
dent of each other. This requires that a single mean or a composite can
be used in a stand-alone comparison only once. For example, when we
compared Group 0 to all of the other groups in combination and if we
were restricting ourselves to orthogonal comparisons, then that would be
the only time that Group 0 could be used alone; it could, however, be part
of a composite of other groups in a different comparison.
There is a limit to the number of orthogonal comparisons that are
possible to perform in any given design. In a one-way design that limit
is equal to the degrees of freedom associated with the between-groups
source of variance, which is a

− 1where a is the number of groups