Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 65

Avoid overrelying on performance metrics when your goal is simply to uncover
basic usability problems. When reporting task success or completion time, it can
be easy to lose sight of the underlying issues behind the data. Performance met-
rics tell the what very effectively but not the why . Performance data can point to
tasks or parts of an interface that were particularly problematic for users, but
they don't identify the causes of the problems. You will usually want to supple-
ment it with other data, such as observational or self-reported data, to better
understand why they were problems and how they might be fixed.

Five basic types of performance metrics are covered in this chapter.

1. Task success is perhaps the most widely used performance metric. It mea-

sures how effectively users are able to complete a given set of tasks. Two
different types of task success are reviewed: binary success and levels of
success. Of course you can also measure task failure.

2. Time on task is a common performance metric that measures how much

time is required to complete a task.

3. Errors reflect the mistakes made during a task. Errors can be useful in

pointing out particularly confusing or misleading parts of an interface.

4. Efficiency can be assessed by examining the amount of effort a user

expends to complete a task, such as the number of clicks in a website or
the number of button presses on a mobile phone.

5. Learnability is a way to measure how performance improves or fails to

improve over time.

4.1 TASK SUCCESS
The most common usability metric is task success, which can be calculated for
practically any usability study that includes tasks. It's almost a universal metric
because it can be calculated for such a wide variety of things being tested—from
websites to kitchen appliances. As long as the user has a reasonably well-defined
task, you can measure success.
Task success is something that almost anyone can relate to. It doesn't require
elaborate explanations of measurement techniques or statistics to get the point
across. If your users can't complete their tasks, then you know something is
wrong. Seeing users fail to complete a simple task can be pretty compelling evi-
dence that something needs to be fixed.
To measure task success, each task that users are asked to perform must have
a clear end state or goal, such as purchasing a product, finding the answer to a
specific question, or completing an online application form. To measure suc-
cess, you need to know what constitutes success, so you should define success
criteria for each task prior to data collection. If you don't predefine criteria, you
run the risk of constructing a poorly worded task and not collecting clean suc-
cess data. Here are examples of two tasks with clear and not-so-clear end states:
•

Findthe5-yeargainorlossforIBMstock(clearendstate)

• Researchwaystosaveforyourretirement(notaclearendstate)