Issue-Based Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 101

often jumping around from one section to another trying to find what they were
lookingfor(thebehavior).”Particularlyifthisissueisfollowedbyavarietyof
detailed examples describing what happened, it could be very helpful. It tells
you where to start looking (the top-level navigation), and the more detailed
examples of additional behaviors may help focus on some possible solutions.
Molich, Jeffries, and Dumas (2007) conducted an interesting study of usability
recommendations and ways to make them more useful and usable. They sug-
gest that all usability recommendations improve the overall user experience of
the application, take into account business and technical constraints, and are
specific and clear.

Of course, not all usability issues are things to be avoided. Some usabil-
ityissuesarepositive.Thesearesometimescalledusability“findings,”asthe
term issues often has negative connotations. Here are some examples of positive
usability issues:

• Allparticipantswereabletologintotheapplication

• Therewerenoerrorsincompletingthesearchtask.

• Participantswerefasteratcreatingareport

The main reason for reporting positive findings, in addition to providing
some positive reinforcement for the project team, is to make sure that these
aspectsoftheinterfacedon'tget“broken”infuturedesigniterations.

5.1.1 Real Issues versus False Issues
One of the most difficult parts of any usability professional's job is determin-
ing which usability issues are real and which are merely an aberration. Obvious
issues are those that most, if not all, participants encounter. For example, it may
be obvious when participants select the wrong option from a poorly worded
menu, get taken down the wrong path, and then spend a significant amount
of time looking for their target in the wrong part of the application. These are
behaviorsthecauseofwhichareusuallya“nobrainer”foralmostanyoneto
identify.

Some usability issues are much less obvious, or it's not completely clear
whether something is a real issue. For example, what if only 1 out of 10 partici-
pants expresses some confusion around a specific piece of content or terminol-
ogy on a website? Or if only 1 out of 12 participants doesn't notice something
she should have? At some point the UX professional must decide whether what
he observed is likely to be repeatable with a larger population. In these situa-
tions, ask yourself whether the participant's behavior, thought process, percep-
tion, or decisions during the task were logical. In other words, is there a consistent
story or reasoning behind her actions or thoughts? If so, then it may be an issue
even if only one participant encountered it. However, no apparent rhyme or rea-
son behind the behavior may be evident. If the participant can't explain why he
did what he did, and it only happened once, then it's likely to be idiosyncratic
and should probably be ignored.