Information Technology Reference
In-Depth Information
that usability issues, and the methods to fix them, are much more complex than
typical bugs. Therefore, it makes more sense to track usability issues in a separate
database. Either way, it's important to track the usability issues and make sure
they are addressed, not simply forgotten.
5.3.4 Some Caveats about Rating Systems
Noteveryonebelievesinseverityratings.Kuniavsky(2003)suggestslettingyour
audience provide their own severity ratings. He argues that only those who are
deeply familiar with the business model will be able to determine the relative
priority of each usability issue.
Bailey(2005)stronglyarguesagainstseverityratingsystemsaltogether.He
cites several studies that show there is very little agreement between usability spe-
cialistsontheseverityratingforanygivenusabilityissue(Catani&Biers,1998;
Cockton&Woolrych,2001;Jacobsen,Hertzum,&John,1998;Molich&Dumas,
2008). All of these studies generally show that there is very little overlap in what
different usability specialists identify as a high-severity issue. Obviously, this is
troubling given that many important decisions may be based on severity ratings.
Hertzum et al. (2002) highlight a potentially different problem in assigning
severity ratings. In their research they found that when multiple usability spe-
cialists are working as part of the same team, each usability specialist rates the
issues she personally identifies as more severe than issues identified by the other
usability specialists on their own team. This is one aspect known as an evaluator
effect, and it poses a significant problem in relying on severity ratings by a single
UX professional. As a profession, we don't yet know why severity ratings are not
consistent between specialists.
So where does this leave us? We believe that severity ratings are far from
perfect, but they still serve a useful purpose. They help direct attention to at
least some of the most pressing needs. Without severity ratings, the designers or
developers will simply make their own priority list, perhaps based on what's eas-
iest or least expensive to implement. Even though there is subjectivity involved
in assigning severity ratings, they're better than nothing. We believe that most
key stakeholders understand that there is more art than science involved, and
they interpret the severity ratings within this broader context.
5.4 ANALYZING AND REPORTING METRICS FOR
USABILITY ISSUES
Once you've identified and prioritized the usability issues, it's helpful to do
some analyses of the issues themselves. This lets you derive some metrics related
to the usability issues. Exactly how you do this will largely depend on the type of
usability questions you have in mind. Three general questions can be answered
by looking at metrics related to usability issues:
•
Howistheoverallusabilityoftheproduct?Thisishelpfulifyousimply
want to get an overall sense of how the product did.
Search WWH ::
Custom Search