Information Technology Reference
In-Depth Information
equal, greater mean differences before it leads you to conclude that the
means are different; it represents a relatively lower level of risk willing
to be taken by researchers in asserting that two means are significantly
different.
Before you commit to your personal choice, recognize that there is no
agreed upon level of power that researchers should wield. Different authors
and different instructors have their own personal viewpoints. It is also not
unreasonable to select a different level of power in different research
situations. As we go along, we will give you more information on which
tobaseyourownchoicesaswellasprovideyouwithsome recommen-
dations.
7.6 ALPHA INFLATION
7.6.1 NUMBER OF COMPARISONS POSSIBLE
As we have suggested in our discussion thus far, it is possible to examine
the results of several comparisons in the post-ANOVA stage of the analysis.
The number of pairwise comparisons that can be made, counting both
the orthogonal and nonorthogonal ones, is obviously a function of the
number of groups (means) that we have in the study; it can be computed
by using the following formula (Cohen, 1996, p. 491):
−
a
(
a
1)
number of possible comparisons
=
.
(7.1)
2
In the above formula,
a
is the number of groups. To illustrate this, if
we had three groups in the study, we could make a total of three pairwise
comparisons: (3
3; if we had five groups in the study,
we could make ten pairwise comparisons: (5
×
2)
÷
2
=
6
÷
2
=
×
÷
=
÷
=
4)
2
20
2
10; if
×
÷
we had seven groups, we could make twenty-one comparisons: (7
6)
2
21; and so forth.
The main statistical drawback to performing all these comparisons is
that if we make enough comparisons at an alpha level of .05, in the long
run some of them are going to be found significant by chance alone if
the null hypothesis is true. In effect, the comparisons are no longer being
made under an alpha level of .05. This is what is meant by
alpha inflation.
If you think of the means from a single study as representing a “family”
of means, then we can talk about the
familywise error
increasing beyond
our stated alpha level. This issue is also known as
cumulative Type I error.
Alpha inflation - an increase in familywise error beyond our alpha level -
can, in turn, lead us to commit one or more Type I (false positive) errors
by asserting that a mean difference is “true” (it reflects a “true” difference
between the conditions) when such is not the case.
Although less pronounced, we still risk some alpha inflation even
when we limit ourselves to making only comparisons that are orthogonal
to each other. With
a
=
42
÷
2
=
−
1 comparisons available to us, we begin on the
