Comparing two designs (or anything else!) using independent sample T-tests - Improving the User Experience through Practical Data Analytics - page 59

FIGURE 2.6
Alternative Excel output.
If we run the test that is called “t-Test: Assuming Unequal Variances,” we get the out-
put in Figure 2.6 . If you do not assume that there are equal variances, the test needs to be
performed a bit differently, and in some sense is not as accurate (lower value of “power”)
and the results (i.e., p -value) change. We do not believe that it is fruitful to go into more
detail about this. If you want to be thorough, you can run each test. The authors have run
many of these tests and, while it is possible, have never found the results to yield a p -value
on one side of 0.05 under one assumption and over 0.05 for the other assumption. As we
will see, in SPSS, the software indicates which of the two tests is appropriate.
Notice that the p -value is 0.0232 (see arrow in Figure 2.6 ), as opposed to the
earlier 0.0246, a difference of 0.0014, a truly negligible difference.
2.5.2 SPSS
Now, we illustrate the same analysis in SPSS. In SPSS, we must type in the data
for both sets of participants into one column, and then designate which data point is
garnered from Design 1 and which data point is garnered from Design 2. So, the data
would be input as in Figure 2.7 .