Combined and Comparative Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 206

Page Visit Efficiency (Target Visits/Actual Visits)

Task 8

Task 7

Task 6

Task 5

Task 4

Task 3

Task 2

Task 1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Figure 8.7 Ratio of target to actual page views for each of the eight tasks.

8.3.2 Comparison to Expert Performance
An alternative to comparing the results of a usability test to predefined goals is
to compare the results to the performance of experts. The best way to determine
the expert performance level is to have one or more presumed experts actually
perform the tasks and to measure the same things that you're measuring in the
usability test. Obviously your experts really need to be experts—people with
subject-matter expertise, in-depth familiarity with the tasks, and in-depth famil-
iarity with the product, application, or website being tested. And your data will
be better if you can average the performance results from more than one expert.
Comparing results of a usability test to results for experts allows you to compen-
sate for the fact that certain tasks may be inherently more difficult or take lon-
ger, even for an expert. The goal, of course, is to see how close the performance
of the participants in the test actually comes to the performance of the experts.

Although you could theoretically do a comparison to expert performance for
any performance metric, it's used most commonly for time data. With task suc-
cess data, the usual assumption is that a true expert would be able to perform all
the tasks successfully. Similarly, with error data, the assumption is that an expert
would not make any errors. But even an expert would require some amount of
time to perform the tasks. For example, consider the task time data shown in
Table 8.10 .