Comparing two designs (or anything else!) using independent sample T-tests - Improving the User Experience through Practical Data Analytics - page 53

satisfactions are different. Thus, the design with the higher ratings of satisfaction is
the winner—and is the one to use for the site's home page.
There are two variations of what the software needs to do in order to correctly
assess the situation and tell us which of the above hypotheses to believe. Either way,
the software does all the work, but the user does need to inform the software which
of the two variations is appropriate. That is, you need to press or check off different
buttons to tell the software what is going on. After all, Excel and SPSS are great, but
they are not mind readers!
In the case of Mademoiselle La La, we'll be performing a “two-sample t-test for
independent samples.” This simply means that for whatever comparison we are mak-
ing, there are two different groups of people involved, each evaluating one of the two
designs . (Not in this case, but in the general case, the two groups might be evaluating
the same design.)
Having two different groups of people is the reason this approach is called “inde-
pendent samples;” no one person is in both groups, and the mean score for one group of
people is totally independent from (i.e., unrelated to) the mean score for the other group.
SIDEBAR: WHY INDEPENDENT GROUPS?
At this point, you may be wondering the reason for using two groups in the irst place. There are
usually two reasons:
First, our goal may be to compare the two groups for which one person cannot be a member of
both groups—for example, the perception of sophistication garnered from women of 18-25 years of
age versus the perception of sophistication of women ranged 26-32 years of age. In that case, there
would likely be one design that each group would experience. (Of course, we can repeat this test
separately for several designs.)
The second reason for having two groups of people would be to compare two designs with
similar groups of people, and it is not appropriate for the same person to experience both designs .
Of course, if it is appropriate for the same person to evaluate both designs, this may be a better
choice—we cover that case in the next chapter (Chapter 3).
Why not? Well, sometimes you'll want to eliminate the “learning curve” from having expe-
rienced the irst design affecting the person, so he/she cannot give an objective evaluation of the
second design.
Here's an analogy: in a medical experiment, you have two different medicines, both of which
might be a safe and effective cure for a medical problem. Although there are rare exceptions
(crossover designs—two or more treatments are applied to the same subject; there is the advantage
of needing fewer subjects, but the disadvantage is that there may be a carryover effect from the irst
treatment administered to the second treatment administered), you would not give the same person
each of the two medications to determine which is more effective, since, after a person has taken
one medication, his/her medical condition has likely changed.
We realize that we're stretching the analogy a little bit, but not too much! Indeed, if the two
designs are like the two medications—in this case, it is the “perspective” or “experience” that has
changed and would not allow an independent evaluation to be made of a second design experienced
by the same person.
The second variation of comparing two samples, called “two-sample t-test with
paired data,” represents the case where the same people are evaluating the two
designs. As we noted earlier, that is the subject of the next chapter.