Database Reference
In-Depth Information
The significance level, as mentioned in the Student's
t
-test discussion, is equivalent
to the type I error. For a significance level such as , if the null hypothesis
( ) is
TRUE
, there is a 5% chance that the observed
T
value based on the
sample data will be large enough to reject the null hypothesis. By selecting an
appropriate significance level, the probability of committing a type I error can be
defined before any data is collected or analyzed.
The probability of committing a Type II error is somewhat more difficult to
determine. If two population means are truly not equal, the probability of
committing a type II error will depend on how far apart the means truly are. To
reduce the probability of a type II error to a reasonable level, it is often necessary
to increase the sample size. This topic is addressed in the next section.
3.3.5 Power and Sample Size
The
power
of a test is the probability of correctly rejecting the null hypothesis. It
is denoted by , where is the probability of a type II error. Because the power
of a test improves as the sample size increases, power is used to determine the
necessary sample size. In the difference of means, the power of a hypothesis test
depends on the true difference of the population means. In other words, for a fixed
significance level, a larger sample size is required to detect a smaller difference in
the means. In general, the magnitude of the difference is known as the
effect size
.
As the sample size becomes larger, it is easier to detect a given effect size, , as
illustrated in
Figure 3.26
.
Figure 3.26
A larger sample size better identifies a fixed effect size
With a large enough sample size, almost any effect size can appear statistically
significant. However, a very small effect size may be useless in a practical sense. It
is important to consider an appropriate effect size for the problem at hand.
