Comparing more than two means: two factor ANOVA with independent samples; the important role of interaction - Improving the User Experience through Practical Data Analytics - page 188

SIDEBAR: THE DANGER OF SEQUENTIAL TESTING—cont'd
This “sequential testing” process occurs frequently in practice. However, there are many reasons
why this is an unwise path forward. The key reason is simply that the conclusions you reach from
two separate one-factor ANOVA studies may be incorrect !
Consider this example: suppose we are examining sophistication ratings, as we did in Chapter 6,
in which the one factor under study was age. Suppose further that for some age groups, females rate
a design as more sophisticated than do males, but for other age groups, males rate the sophistication
level to be higher than do females. In other words, it is possible that, averaged over all the ages ,
males and females give about the same sophistication ratings, so it looks like gender has no effect
on sophistication rating, and, averaged over both genders , perhaps all the age groups get about the
same sophistication rating, so it appears that there is no difference there either.
But you would be missing that fact that for some ages, there is a large gender gap in rating in
one direction, while for other ages, there is a large gender gap in the other direction!! A prudent
experiment would allow for the measurement of this possibility, which we (and much of the rest of
the world!) refer to as interaction . We will discuss this notion at length later in the chapter.
8.2 CASE STUDY: COMPARING AGE AND GENDER
AT MADEMOISELLE LA LA
Let's return yet again to our favorite fashion site, Mademoiselle La La. To refresh
your memory, you were hired as the UX researcher at Mademoiselle La La, a high-
end online store aimed at urbane women from ages 18-55 years with well-above-
average disposable income.
In Chapter 6, you'll remember that Cinny Bittle, your director of marketing, got
her hand on a Forrester report that claimed older boomers (aged 56-66) spend the
most online of all generations. Since Bittle was worried that your new home page
design was not considered sophisticated by this age bracket, you offered to slice and
dice the survey data by age. You were trying to determine if there were different
perceptions of sophistication by age. To refresh your memory, you sorted the data by
youngest to oldest, using these age brackets:
1. Gen Z, 18-25
2. Gen Y, 26-35
3. Gen X, 36-45
4. Younger boomers, 46-55
5. Older boomers, 56-66
Then you calculated the mean for each age bracket, as displayed in Table 8.1 .
With your basic work out of the way, you performed your ANOVA analysis and
Student-Newman-Keuls (S-N-K) test, and you determined that the means of age
groups 2 and 3 (Gen Y [26-35] and Gen X [36-45]) cannot be said to be different,
but the means of those two age groups can be said to be higher than the means of the
other three age groups.