Issue-Based Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 119

• ItisOKtomisssomeofthemajorusabilityissues.Youaremoreinter-

ested in capturing some of the big issues, iterating the design, and retest-
ing. Any improvement is welcome.

• Thereisonlyonedistinctusergroupthatyoubelievewillthinkabout

the design and tasks in a very similar way.

• Thescopeofthedesignislimited.Thereareasmallnumberofscreens

and/ortasks.

Werecommendincreasingthenumberofparticipantsand/orthenumberof
UX teams when the following conditions apply:

• YoumustcaptureasmanyUXissuesaspossible.Inotherwords,there

will be significant negative repercussions if you miss any of the major
usability issues.

• Thereismorethanonedistinctusergroup.

• Thescopeofthedesignislarge.Inthiscasewewouldrecommenda

broader set of tasks.

We fully realize that not everyone has access to multiple UX researchers. In
thiscase,trytosolicitfeedbackfromanyotherobservers.Noonecanseeevery-
thing. Also, you might want to acknowledge that some of the major usability
issues might not have been identified.

5.8 SUMMARY
Many usability professionals make their living by identifying usability issues
and by providing actionable recommendations for improvement. Providing
metrics around usability issues is not commonly done, but it can be incorpo-
rated easily into anyone's routine. Measuring usability issues helps you answer
some fundamental questions about how good (or bad) the design is, how it is
changing with each design iteration, and where to focus resources to remedy
the outstanding problems. You should keep the following points in mind when
identifying, measuring, and presenting usability issues.

1. The easiest way to identify usability issues is during an in-person lab
study, but it can also be done using verbatim comments in an automated
study. The more you understand the domain, the easier it will be to spot
the issues. Having multiple observers is very helpful in identifying issues.
2. When trying to figure out whether an issue is real, ask yourself whether
there is a consistent story behind the user's thought process and behav-
ior. If the story is reasonable, then the issue is likely to be real.
3. Theseverityofanissuecanbedeterminedinseveralways.Severityalways
should take into account the impact on the user experience. Additional
factors, such as frequency of use, impact on the business, and persistence,
mayalsobeconsidered.Someseverityratingsarebasedonasimplehigh/
medium/lowratingsystem.Othersystemsarenumberbased.

4. Some common ways to measure usability issues are measuring the fre-

quency of unique issues, the percentage of participants who experience