Comparing more than two means: one factor ANOVA with independent samples. Multiple comparison testing with the Newman-Keuls test - Improving the User Experience through Practical Data Analytics - page 150

FIGURE 6.7
One-way analysis of variance setup for obtaining output; SPSS.
The most common way to determine which of the column/treatment/design (true)
means differ from others is to perform a multiple comparison test. This means, essentially,
that you compare every column mean with every other column mean (hence the name—
there are multiple comparisons being made). If there are 4 columns, there are 6 combina-
tions of 2 at a time; if there are 5 columns, there are 10 combinations of 2 at a time.
SIDEBAR: WHICH MULTIPLE COMPARISON TEST SHOULD YOU USE?
There are several multiple comparison tests; each has somewhat different properties, with some of
them including different assumptions. Also, reasonable people disagree on which is the best of these
multiple comparison tests. In fact, most often, the answer will be the same regardless of which test
is used (keeping the p -value cutoff point constant, say, at the usual 0.05). Sometimes, different tests
will yield slightly different answers, but it would be infrequent to ind very different answers.
We have chosen to illustrate how to use only one test, the one we believe is the best choice—the
Newman-Keuls test. Sometimes, this test is referred to as the Student-Newman-Keuls test, and is
called the “S-N-K” test in SPSS. We will, henceforth, refer to it as the S-N-K test, since SPSS will
be critical to our inding the results of the test. It is not that critical for you to dive into the intimate
details of the test—although we will give a very brief outline of how it works.