Information Technology Reference
In-Depth Information
enough to fall in the 5 percent region of the particular statistic's sampling
distribution and so we judge that difference to be statistically significant;
this other mean difference is too small, falling in the 95 percent region
of the particular statistic's sampling distribution, and so we judge that
difference to not be statistically significant, and so on.
Here is the conundrum: statistical power is the ability to find an effect
if it exists, but we cannot know truly whether the effect actually exists. We
therefore apply the concept of power by estimating the relative likelihood
of this multiple comparison procedure versus that multiple comparison
procedure yielding a statistically significant outcome for a given mean
difference with all else equal. And, just in case you are now asking the
question, the answer is yes, these tests will from time to time provide
different outcomes. And, of course, there is no way to know which ones
are “right” when they do yield different results. So we do not worry about
it. Instead, we say that Multiple Comparison Procedure X has more power
than Multiple Comparison Procedure Y, meaning that Procedure X is
more likely than Procedure Y to return a significant result given a certain
mean difference with all else equal. An alternative way to view this is that
Procedure X is the more powerful of the two if it will find a given mean
difference to be statistically significant when Procedure Y will not result
in obtaining a significant difference.
7.5.3 THE CONTINUUM OF POWER
At the higher end of the power continuum is, to use colloquial language,
lots of power. A procedure near the higher power end of the continuum is
more likely than most to yield a significant difference for a given magnitude
of mean difference (all else equal). At the other end is low power. A
procedure near the lower power end of the continuum is less likely than
most to yield a significant difference for a given magnitude of mean
difference.
At least in the world of research and statistics, we generally have a
positive regard for power. Who would not admire a procedure that could
find an effect that was actually there? But here, as in the general case of
power, there are risks as well as benefits. The very powerful procedures -
those more likely to find smaller differences to be statistically significant -
run an increased risk of leading us to make a false positive (Type I) error
(we falsely conclude that a mean difference is statistically significant).
Given the relative ease of finding statistical significance with more
powerful tests, another term that can be used to indicate power is the term
liberal . A test is said to be more liberal (powerful) when it can more easily
yield statistically significant mean differences; use of such a test represents
a relatively greater amount of risk willing to be taken by researchers in
asserting that two means are significantly different. The corresponding
label that is used for other end of the continuum is conservative .Atestis
said to be more conservative (less powerful) when it is less likely to evaluate
a difference as statistically different. A conservative test needs, all else being
Search WWH ::




Custom Search