Information Technology Reference
In-Depth Information
4.7.2 POPULATION EFFECT SIZE
Effect size
, in this context, is the mean difference of the groups in the pop-
ulation evaluated against the within-groups population variance. What
represents a large effect size has to do with the phenomenon under study,
which is by its nature weak, moderate, or strong. There is nothing we
researchers can do to affect this reality; rather, it is our responsibility to
measure the phenomenon as accurately and objectively as possible and
to let the chips fall where they may. The larger the population effect size,
that is, the more potent the independent variable is in distinguishing the
population group means against a background of error, the greater power
we will have in the study.
On the other hand, some researcher judgment may be involved in
selecting the levels of the independent variable to use in a study or even
in selecting the best measure of behavior to use for the study at hand.
For example, if we wanted to determine if preparation time enhanced
performance on a final exam of a particular course, we might find a group
of students who did not study at all to serve in the control group. If we
chose to recruit in an experimental group those who actually studied
for half an hour (as a very exaggerated example to make the point), the
likelihood of finding an effect of the independent variable (believing very
strongly that study time does truly matter) is practically nil, despite the
fact that the population effect size is large. Rather, we would surely want
to increase the power of our study by selecting an amount of study time
for our experimental group that should show a difference if in fact our
intuition was correct.
4.7.2 SAMPLE SIZE
Finally, sample size is an important factor in statistical power. The larger
the sample size, the greater will be our statistical power. This is true at a
basic level in that the closer we come to sampling the entire population, the
more precise an estimate we will be able to make of the population mean
for each group. At its extreme, if we sampled the entire population, and
if we found a difference between the means of the two groups, we would
not need a statistical test to determine if they were significantly different -
by definition, the means would be statistically different.
In some fields of research, it is common to work with large databases,
such as those used in archival research and in educational research. With
tens or sometimes hundreds of thousands of cases, virtually any mean
difference will turn out to be statistically significant. In such research,
criteria other than statistical significance, such as strength of effect, must
be used to evaluate the findings lest the researchers be overwhelmed by
the amount of statistical power they wield. With very large sample sizes
(e.g.,
N
1, 000), it is common practice to focus very little on statistical
significance and to focus much more on the magnitudes of the effects
under study. In fact, there has been a movement in many disciplines to
≥
