Self-Reported Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 122

6.7.4 Awareness and Comprehension 159

6.7.5 Awareness and Usefulness Gaps

160

6.8 SUMMARY 161

Perhaps the most obvious way to learn about the usability of something is to
ask the participants to tell you about their experience with it. But exactly how to
ask participants so that you get good data is not so obvious. The questions you
might ask could take on many forms, including various kinds of rating scales,
lists of attributes that the participants choose from, and open-ended questions
such as “List the top three things you liked the most about this application.”
Some of the attributes you might ask about include overall satisfaction, ease of
use, effectiveness of navigation, awareness of certain features, clarity of termi-
nology, visual appeal, trust in a company that sponsors a website, enjoyment
in playing a game, and many others. But the common feature of all of these is
you're asking the participant for information, which is why we think self-reported
best describes these metrics. And as we will see, one critical type of self-reported
data is the verbatim comments made by participants while using a product.

THE EVOLUTION OF USABILITY AND USER EXPERIENCE

One of the historical precedents for the usability field was human factors, or ergonomics,
which itself grew primarily out of World War II and a desire to improve airplane cockpits
to minimize pilot error. With this ancestry, it's not surprising that much of the early
focus of usability was on performance data (e.g., speed and accuracy). But that has been
changing, quite significantly we think. Part of the reason for the widespread adoption
of the term “user experience,” or UX, is the focus that it provides on the entire range of
experience that the user has with a product. Even the Usability Professionals Association
changed its name in 2012 to the User Experience Professionals Association. All of this
reflects the importance of the kind of metrics discussed in this chapter, which try to
encompass such states as delight, joy, trust, fun, challenge, anger, frustration, and many
more. An interesting analysis was done by Bargas-Avila and Hornbæk (2011) of 66
empirical studies in the UX literature from 2005 to 2009 showing how the studies reflect
some of these shifts. They found, for example, that emotions, enjoyment, and aesthetics
were the most frequently assessed UX dimensions in the recent studies.

Two other terms sometimes used to describe this kind of data include sub-
jective data and preference data. Subjective is used as a counterpart to objective ,
which is often used to describe performance data from a usability study. But this
implies that there's a lack of objectivity to the data you're collecting. Yes, it may
be subjective to each participant who's providing the input, but from the per-
spective of the user experience professional, it is completely objective. Similarly,
preference is often used as a counterpart to performance . Although there's nothing
obviously wrong with that, we believe that preference implies a choice of one
option over another, which is often not the case in UX studies.