Combined and Comparative Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 188

ways you can combine or represent different metrics to get an overall view of the
usability of a product, or different aspects of a product, perhaps as revealed by
different tasks.

The most common question asked after a usability test is “How did it do?”
People who ask this question (often the product manager, developer, or other
members of the project team) usually don't want to hear about task completion
rates, task times, or questionnaire scores. They want an overall score of some type:
Did it pass or fail? How did it do in comparison to the last round of usability test-
ing? Making these kinds of judgments in a meaningful way involves combining
the metrics from a usability test into some type of overall score. The challenge
is figuring out how to combine scores from different scales in a meaningful way
(e.g., task completion rates in percentages and task times in minutes or seconds).

8.1.1 Combining Metrics Based on Target Goals
Perhaps the easiest way to combine different metrics is to compare each data
point to a target goal and represent one single metric based on the percentage of
users who achieved a combined set of goals. For example, assume that the goal
is for users to complete at least 80% of their tasks successfully in no more than
70 seconds each on the average. Given that goal, consider the data in Table 8.1 ,
which shows task completion rate and average time per task for each of eight
participants in a usability test.

Table 8.1 shows some interesting results. The average values for task comple-
tion (82%) and task time (67 seconds) would seem to indicate that the goals for
this test were met. Even if you look at the number of users who met the task com-
pletion goal (six participants, or 75%) or the task time goal (five participants,
or 62%), you would still find the results reasonably encouraging. However, the
most appropriate way to look at the results is to see if each individual participant

Table 8.1 Sample task completion and task time data from eight participants a .
Participant # Task Completion Task Time (secs) Goal Met?
1 85% 68 1
2 70% 59 0
3 80% 79 0
4 75% 62 0
5 90% 72 0
6 80% 60 1
7 80% 56 1
8 95% 78 0
Average: 82% 67 38%
a Also shown are averages for task completion and time and an indication of whether each participant met the
objective of completing at least 80% of the tasks in no more than 70 seconds.

Table 8.1 Sample task completion and task time data from eight participants a .
a Also shown are averages for task completion and time and an indication of whether each participant met the
objective of completing at least 80% of the tasks in no more than 70 seconds.