Combined and Comparative Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 187

CHAPTER 8
Combined and
Comparative Metrics

187

CONTENTS
8.1 SINGLE USABILITY SCORES

187

8.1.1 Combining Metrics Based on Target Goals 188

8.1.2 Combining Metrics Based on Percentages 189

8.1.3 Combining Metrics Based on Z Scores 196

8.1.4 Using Single Usability Metric 198

8.2 USABILITY SCORECARDS 200

8.3 COMPARISON TO GOALS AND EXPERT PERFORMANCE 204

8.3.1 Comparison to Goals 204

8.3.2 Comparison to Expert Performance 206

8.4 SUMMARY 208

Usability data are building blocks. Each piece of usability data can be used to
create new metrics. Raw usability data can include task completion rates, time
on task, or self-reported ease of use. All of these usability data can be used to
derive new metrics that were not available previously, such as an overall usabil-
ity metric or some type of “usability scorecard.” Why might you want to do this?
We think the most compelling reason is to have an easy-to-understand score or
summary of all the metrics you've collected in a study. This can be very handy
when presenting to senior managers, for tracking changes across iterations or
releases, and for comparing different designs.

Two of the common ways to derive new usability metrics from existing data
are by (1) combining more than one metric into a single usability measure and
(2) comparing existing usability data to expert or ideal results. Both methods are
reviewed in this chapter.

8.1 SINGLE USABILITY SCORES
In many usability tests, you collect more than one metric, such as task comple-
tion rate, task time, and perhaps a self-reported metric such as a system usabil-
ity scale (SUS) score. In most cases, you don't care so much about the results
for each these metrics individually as you do about the total picture of the user
experience as reflected by all of these metrics. This section covers the various