Database Reference
In-Depth Information
6.8.1
SUPPOSE THAT YOUR ONLY SOFTWARE AVAILABLE IS EXCEL
SIDEBAR: HOW TO PERFORM MULTIPLE MEANS TESTING IN EXCEL
(WITH CAVEATS!)
If you don't have SPSS, you can still do multiple comparison testing in Excel after you've run
your ANOVA. You simply run a t-test on the two means you are interested in. Using the example in
this chapter, you may be wondering if there really is a statistical difference of mean sophistication
ratings between the older and younger boomers. Since these are independent samples, you would
use the independent t-test as described in Chapter 2. However, you should be aware that there can
be bias induced when you decide on testing various pairs of means, if the choices of which pairs of
means to t-test are based on seeing what the values of actual sample means are. This may sound a
bit strange, but if you decide on which pairs of means to test before you see any results, that's ine
(unbiased!!).
This potential bias is corrected for in the S-N-K test, where all pairs are tested.
However, things get a little more complicated if you want to test more than one pair of means.
For example, say you want to perform an independent t-test between older and younger boomers,
and then an independent t-test between Gen X and Gen Y—both tests decided to be done before
seeing the actual means of the age-groups. When you do this, the probability of making a type I
error (or α) increases. After all, if each test has α = 0.05, then, because there are two opportunities
(i.e., two t-tests) with potential to make a type I error, the probability of making at least one of these
errors in the two opportunities to do so is more than 0.05. Without getting too technical, the prob-
ability is not exactly, but close to, twice as high. Just think of lipping 10 coins. The chance of 10
heads in a row is about 0.001; however, if you tried the task 500 times, the chances are a lot higher
(about 39%) that you get 10 heads in a row
at least once
!!!
To alleviate this problem and retain at approximately 0.05 the probability of making at least one
type I error, thus coming closer to duplicating (but not totally duplicating) the logic of the S-N-K
test, a good rule of thumb is to choose as a value of α for each t-test you perform a value which is
different from 0.05. Instead, you should choose as α for each t-test (0.05 ÷ [the number of t-tests
you're performing]). So, in this case of performing two t-tests, you would use 0.025 as the α for
each t-test. If you're performing three t-tests, you should divide 0.05 by 3, which is 0.0167, and use
0.0167 as your new alpha for each t-test. And so on.
As we noted, in SPSS, your multiple comparison testing takes care of all the issues alluded to
above (potential biases, changing the value of α, etc.), so you don't need to worry about any of these
issues. Do you see SPSS in your future?
SIDEBAR: WHY CAN'T I SKIP THE ANOVA STEP?
You may be wondering why you can't skip ANOVA altogether and go right to multiple compari-
son testing by performing either an S-N-K in SPSS or multiple t-tests in Excel. “ANOVA doesn't
tell me exactly how the means differ,” you may be saying to yourself. “It tells me only that the
population means are either all the same or not all the same. What good is that? I want to know how
speciic means differ. Why can't I just skip ANOVA?”
The answer in a word, again, is “bias.” Suppose that you have several men from each state in
the US in a (large) room. If you consider the highest state mean and the lowest state mean, that
difference will be relatively large, even when the true mean for each state is the same!! Say, just by
chance, the two extreme means are from Louisiana and Idaho. If you do a two-sample t-test on the
means of those two states, it will extremely likely be highly signiicant (because it is the biggest
difference among 50 different means!!). Is that a reason to conclude that men from those states
Continued
Search WWH ::
Custom Search