Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 92

This gives a very straightforward efficiency score for each participant: number of
tasks completed successfully per minute (or whatever unit of time you used).
If a participant completed 10 tasks successfully in a total time of 10 minutes,
then that participant was successfully completing 1 task per minute overall. This
works best when all participants attempted the same number of tasks and the
tasks are relatively comparable in terms of their level of difficulty.

Figure 4.10 shows data from an online study comparing four different navi-
gation prototypes for a website. This was a between-subjects study, in which
each participant used only one of the prototypes, but all participants were asked
to perform the same 20 tasks. Over 200 participants used each prototype. We
were able to count the number of tasks completed successfully by each par-
ticipant and divide that by the total time that participant spent. The averages of
these (and the 95% confidence intervals) are shown in Figure 4.10 .

Average Efficiency (Tasks Successfully Completed per Minute)

1.65

1.60

1.55

1.50

1.45

1.40

1.35

1.30

1.25

1.20

Prototype 1 Prototype 2 rototype 3 rototype 4

Figure 4.10 Average number of tasks completed successfully per minute in an online study of four
different prototypes of navigation for a website. Over 200 participants attempted 20 tasks with each
prototype. Participants using Prototype 2 were significantly more efficient (i.e., completed more tasks per
minute) than those using Prototype 3.

4.5 LEARNABILITY
Most products, especially new ones, require some amount of learning. Usually,
learning does not happen in an instant but occurs over time as experience
increases. Experience is based on the amount of time spent using a product and
the variety of tasks performed. Learning is sometimes quick and painless, but at
other times it is quite arduous and time-consuming. Learnability is the extent
to which something can be learned efficiently. It can be measured by looking
at how much time and effort are required to become proficient, and ultimately
expert in using something. We believe that learnability is an important user