Comparing more than two means: two factor ANOVA with independent samples; the important role of interaction - Improving the User Experience through Practical Data Analytics - page 203

“Oh, don't bother,” she says, waving her hand in the air. She leans in, partially
covers her mouth, and whispers: “This is tremendous work, but Cinny doesn't really
grasp this kind of thing, and actually gets lummoxed when I spend too much time
asking about your research.”
“Oh?” you ask, trying to retain objectivity.
She leans in, puts her hand up to mouth and whispers: “I think she's intimidated
by your statistical savoir-faire . Let's just keep it between us. Deal?”
“Deal!”
“Thanks again,” she says as she leaves. “The board is gonna love this!”
SIDEBAR: REPLICATION: THE GOOD, THE BAD AND THE … “PENALTY”.
The issue of replication and the role it plays in our ability to measure interaction effects is important
to understand.
We have gone out of our way to separate the two-factor design, as covered in this chapter, from
the situation in Chapter 7, where you have two factors, but one of them is simply, “respondent” (or,
equivalently, “participant”), which can be considered a “faux factor.”
If the two factors are ones that do not include “participant” (a random level factor as discussed
in Chapter 7), which is the case in this chapter - Age and Gender are the two factors - it is very
likely you will have replication, and not just one data point for each combination.
For example, with 5 age groups and 2 genders (using the data from this chapter), it would hard
to imagine having only two respondents for each age group - one male and one female. Thus, there
is replication. Indeed, in a case not involving “respondent” as a factor, the “no replication” situation
is very unlikely to arise.
However, when you have two factors and one is “respondent” (making it a within-subjects
design), and you have only one “real” factor, then you will not have replication.
This is what occurred in the previous chapter, Chapter 7, when you needed to perform a few
added steps, clicking on “Model” and then “Customization.” Basically, you were telling SPSS that
there was no interaction between “person” and “task.” This was necessary because there is, indeed,
no replication in that example - for any individual participant and task combination, there was only
one data value.
Here's the bottom line:
If you do not have replication, you must assume no interaction with two ixed-level factors.
Why is this important? Well, irst of all, you need to tell the software whether or not there is
interaction so that the software will perform the analysis correctly.
But there's another reason: there may be what we call an “analysis penalty.” That is, in running
an experiment with no replication (for example, because you want to save time and money) means
you MUST assume that there is no interaction.
Why is that a “penalty”? Well, there might indeed be an interaction effect between the factors,
but you MUST assume that there is no interaction, and there is no way to know if the “no interac-
tion” assumption is “doing you in!” You can think of the situation as analogous to the old saying,
“There's no free lunch.” ☺
But here's the good news: you can comfortably assume no interaction when one of the factors
is “respondent.” And that's indeed the case with the standard “within-subjects” usability test
described in Chapter 7.
Nevertheless, as noted, you still need to tell the software there is no interaction. The way you do
this is to repeat the process as we discussed in Chapter 7, illustrated by Figures 7.8 through 7.12 and
the accompanying text.
Continued