The Statistical Significance of F and Effect Strength - Analysis of Variance Designs - page 41

rejected the null hypothesis. However, here when we compute the ratio of
between-groups to within-groups variance, the F ratio is not large enough
to fall into the 5 percent area under the curve. Thus, we fail to reject
the null hypothesis and claim instead that the means are not significantly
different. A Type II error is therefore a false negative judgment concerning
the validation of the mean difference obtained. There are several possible
reasons that may account for why a Type II error was made. Among these
reasons is that we had insufficient statistical power in the study to detect
a true difference between or among the means; this topic is discussed in
Section 4.7.
4.5 EXPLAINING VARIANCE: STRENGTH OF EFFECT
4.5.1 OVERVIEW
We infer from a statistically significant F ratio that the difference between
or among the means of the levels of the independent variable is reliably
different. We can therefore assert (with 95 percent confidence) that the
independent variable did distinguish our groups from each other on the
dependent measure. Using our room color study as an example, we con-
cluded that room color seemed to affect the mood of the students. But the
F ratio does not inform us of the strength (magnitude, potency) of the
effect of the independent variable.
One way to conceptualize what is meant by “strength of effect” is
to recognize that if the means are significantly different, then we can
predict or differentiate with better-than-chance likelihood the scores of
the participants on the dependent variable given the group to which
they belong. But one can defeat the chance odds by a little, a moderate
amount, or by a large margin. The degree to which we can perform such a
prediction or differentiation is indexed by the degree to which the scores
on the dependent variable are associated with the particular levels of the
independent variable.
Strength (or magnitude) of effect measures are indexes of how strongly
the scores and the levels of the independent variable are related to each
other. The two most commonly used indexes evaluating the strength of the
effect are omega squared and eta squared indexes (Keppel, 1991; Keppel &
Wickens, 2004).
4.5.2 OMEGA SQUARED
Omega squared represents the strength of the effect of the independent
variable in the population. Because we rarely know the population param-
eters needed to perform that calculation, we ordinarily estimate omega
squared based on the results of the ANOVA. The formula for this calcula-
tion can be found in Keppel's (1991) text. Whereas omega squared varies
between zero and one, the estimate of omega squared can yield a negative
value when the F ratio is less than one.