Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 91

FormatforUsabilityTestReports(ISO/IEC25062:2006)specifiesthatthe“core
measure of efficiency” is the ratio of the task completion rate to the mean time
per task. Basically, it expresses task success per unit time. Most commonly, time
per task is expressed in minutes, but seconds could be appropriate if the tasks
are very short or even hours if they are unusually long. The unit of time used
determines the scale of the results. Your goal is to choose a unit that yields a
“reasonable” scale (i.e., one where most of the values fall between 1 and 100%).
Table 4.3 shows an example of calculating an efficiency metric based on task
completion and task time. Figure 4.9 shows how this efficiency metric looks in
a chart.

Table 4.3 The efficiency measure is simply the ratio of task completion to task time in minutes a .
Task Completion Rate Task Time (min) Efficiency (%)
Task 1 65% 1.5 43
Task 2 67% 1.4 48
Task 3 40% 2.1 19
Task 4 74% 1.7 44
Task 5 85% 1.2 71
Task 6 90% 1.4 64
Task 7 49% 2.1 23
Task 8 33% 1.3 25
a Of course, higher values of efficiency are better. In this example, users appear to have been more efficient in
performing tasks 5 and 6 than the other tasks.
a Of course, higher values of efficiency are better. In this example, users appear to have been more efficient in
performing tasks 5 and 6 than the other tasks.

Table 4.3 The efficiency measure is simply the ratio of task completion to task time in minutes a .

Efficiency (Task Success per Minute)

80%

70%

60%

50%

40%

30%

20%

10%

0%

Task 1Task 2Task 3Task 4Task 5Task 6Task 7Task 8

Figure 4.9 An example showing efficiency as a function of completion rate/time.

A slight variation on this approach to calculating efficiency is to count the
number of tasks completed successfully by each participant and divide that by
the total time spent by the participant on all tasks (successful and unsuccessful).