Background - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 31

coefficient is a measure of the strength of the relationship between the two variables and
has a range from −1 to +1. The stronger the relationship, the closer the value is to −1 or +1.
The weaker the relationship, the closer the correlation coefficient is to 0. The negative value
for “r” signifies the negative relationship between the two variables. If you square the cor-
relation coefficient you get the same value as the R 2 value shown on the scatterplot (0.28).
2.6 NONPARAMETRIC TESTS
Nonparametric tests are used for analyzing nominal and ordinal data. For exam-
ple, you might want to know if a significant difference exists between men and
women for success and failure on a particular task. Or perhaps you're interested
in determining whether there is a difference among experts, intermediates, and
novices on how they ranked different websites. To answer questions that involve
nominal and ordinal data, you will need to use some type of nonparametric test.
Nonparametric statistics make different assumptions about the data than
the statistics we've reviewed for comparing means and describing relationships
between variables. For instance, when we run t tests and correlation analysis, we
assume that data are distributed normally and the variances are approximately
equal. The distribution is not normal for nominal or ordinal data. Therefore, we
don't make the same assumptions about the data in nonparametric tests. For
example, in the case of (binary) success, when there are only two possibilities,
the data are based on the binomial distribution. Some people like to refer to
nonparametric tests as “distribution-free” tests. There are a few different types of
nonparametric tests, but we will just cover the χ 2 test because it is probably the
most commonly used.
2.6.1 The χ 2 Test
The χ 2 (pronounced “chi square”) test is used when you want to compare nomi-
nal (or categorical) data. Let's consider an example. Assume you're interested
in knowing whether there is a significant difference in task success among three
different groups: novice, intermediates, and experts. You run a total of 60 people
in your study, 20 in each group. You measure task success or failure on a single
task. You count the number of people who were successful in each group. For
novices, only 6 out of 20 were successful, 12 out of 20 intermediates were suc-
cessful, and 18 out of 20 experts were successful. You want to know if there is a
statistically significant difference among the groups.
EXCEL TIP
To perform a χ 2 test in Excel, you use the “CHITEST” function. This function calculates
whether differences between observed and expected values are simply due to chance.
The function is relatively easy is to use:
= CHITEST(actual_range, expected_range)