Combined and Comparative Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 192

may indicate a different weighting. In this example, we're combining two per-
formance measures (task completion and task time) with one self-reported mea-
sure (rating). By giving equal weight to each, we're actually giving twice as much
weight to performance as to the self-reported measure. That can be adjusted by
using weights in calculating the averages, as shown in Table 8.4 .
Table 8.4 Calculation of weighted averages a .

Weighted
Average
1 38% 1 47% 1 60% 2 51%
2 50% 1 60% 1 65% 2 60%
3 74% 1 87% 1 78% 2 79%
4 36% 1 40% 1 43% 2 40%
5 89% 1 73% 1 80% 2 81%
6 48% 1 60% 1 83% 2 68%
7 43% 1 53% 1 63% 2 55%
8 42% 1 47% 1 35% 2 40%
9 100% 1 60% 1 95% 2 88%
10 45% 1 67% 1 90% 2 73%
a Each individual percentage is multiplied by its associated weight, these products are summed, and that sum is divided by the sum of the
weights (4, in this example).

Participant # Time Weight Tasks Weight Rating Weight

Table 8.4 Calculation of weighted averages a .
a Each individual percentage is multiplied by its associated weight, these products are summed, and that sum is divided by the sum of the
weights (4, in this example).

In Table 8.4 , the subjective rating is given a weight of 2, and each of the two
performance measures is given a weight of 1. The net effect is that the subjec-
tive rating gets as much weight in the calculation of the average as the two per-
formance measures together. The result is that these weighted averages for each
participant tend to be closer to the subjective ratings than the equal-weight aver-
ages in Table 8.3 . The exact weights you use for any given product should be
determined by the business goals for the product. For example, if you're testing
a website for use by the general public, and the users have many other com-
petitors' websites to choose from, you might want to give more weight to self-
reported measures because you probably care more about the users' perception of
the product than anything else.

However, if you're dealing with an application where speed and accuracy are
more important, such as a stock-trading application, you would probably want
to give more weight to performance measures. You can use any weights that
are appropriate for your situation, but remember to divide by the sum of those
weights in calculating the weighted average.

These basic principles apply to transforming any set of metrics from a usabil-
ity test. For example, consider the data in Table 8.5 , which includes number of
tasks completed successfully (out of 10), number of web page visits, an overall
satisfaction rating, and an overall usefulness rating.