Issue-Based Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 100

Usability issues are generally thought of as purely qualitative. They typically
include the identification and description of a problem one or more partici-
pants experienced and, in many cases, an assessment of the underlying cause of
the problem. Most UX professionals also include specific recommendations for
remedying the problem and many report positive findings as well (i.e., some-
thing that worked particularly well).

Most UX professionals don't strongly associate metrics with usability issues.
This may be because of the gray areas in identifying issues or because identify-
ing issues is part of an iterative design process, and metrics are perceived as add-
ing little value. However, not only is it possible to measure usability issues, but
doing so also adds value in product design while not slowing down the iterative
design process.

This chapter reviews some simple metrics around usability issues. It also dis-
cusses different ways of identifying usability issues, prioritizing the importance
of different types of issues, and factors you need to think about when measuring
usability issues.

5.1 WHAT IS A USABILITY ISSUE?
What do we mean by usability issues? Usability issues are based on behavior
in using a product. As a UX professional you interpret the cause of these issues,
such as confusing terminology or hidden navigation. Examples of the more
common types of usability issues include:

• Behaviorsthatpreventtaskcompletion

• Behaviorsthattakessomeone“offcourse”

• Anexpressionoffrustrationbytheparticipant

• Notseeingsomethingthatshouldbenoticed

• Aparticipantsaysataskiscompletewhenitisnot

• Performinganactionthatleadsawayfromtasksuccess

• Misinterpretingsomepieceofcontent

• Choosingthewronglinktonavigatethroughwebpages

A key point to consider in defining usability issues is how they will be
addressed. The most common use is in an iterative design process focused on
improving the product. In that context, the most useful issues are those that
point to possible improvements in the product. In other words, it helps if issues
are reasonably actionable. If they don't point directly to a part of the interface
that was causing a problem, they should at least give you some hint of where to
begin looking. For example, we once saw an issue in a usability test report that
said,“Thementalmodeloftheapplicationdoesnotmatchtheuser'smental
model.”Notethatnobehaviorwasmentioned.Andthatwasit.Althoughthis
may be an interesting interpretation of some behavior in a theoretical sense, it
does very little to guide designers and developers in addressing the issue.

However,consideranissuelikethis:“Manyparticipantswereconfusedby
the top-level navigation menu (which is the interpretation of the behavior),