Combined and Comparative Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 189

met the stated goal (i.e., the combination of completing at least 80% of the tasks
in no more than 70 seconds each). It turns out, as shown in the last column of
Table 8.1 , that only three, or 38%, of the participants actually met the goal. This
demonstrates the importance of looking at individual participant data rather
than just looking at averages. This can be particularly true when dealing with
relatively small numbers of participants.

This method of combining metrics based on target goals can be used with
any set of metrics. The only real decision is what target goals to use. Target goals
can be based on business goals and/or comparison to ideal performance. The
math is easy (each person just gets a 1 or 0), and the interpretation is easy to
explain (the percentage of users who had an experience that met the stated goal
during the test).

8.1.2 Combining Metrics Based on Percentages
Although we're well aware that we should have measurable target goals for our
usability tests, in practice many of us don't have them. So what can you do to
combine different metrics when you don't have target goals? One simple tech-
nique for combining scores on different scales is to convert each score to a per-
centage and then average them. For example, consider the data in Table 8.2 ,
which shows results of a usability test with 10 participants.

One way to get an overall sense of the results from this study is to first convert
each of these metrics to a percentage. In the case of the number of tasks com-
pleted and the subjective rating, it's easy because we know the maximum (“best”)
possible value for each of those scores: there were 15 tasks, and the maximum

Table 8.2 Sample data from a usability test with 10 participants a .

Tasks Completed
(of 15) Rating (0-4)
1 65 7 2.4
2 50 9 2.6
3 34 13 3.1
4 70 6 1.7
5 28 11 3.2
6 52 9 3.3
7 58 8 2.5
8 60 7 1.4
9 25 9 3.8
10 55 10 3.6
a Time per task is the average time to complete each task, in seconds. Tasks completed are number of tasks (out
of 15) that the user completed successfully. Rating is the average of a five-point task ease rating for each task,
where higher is better.

Time Per Task
(sec) Participant #

Table 8.2 Sample data from a usability test with 10 participants a .
a Time per task is the average time to complete each task, in seconds. Tasks completed are number of tasks
(out of 15) that the user completed successfully. Rating is the average of a five-point task ease rating for each
task, where higher is better.