Introduction - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 13

collecting task time data or any other type of usability data. There are some
simple things that can be done to minimize or even remove noise in the data.
UX data can be cleaned up so that extreme values are not used in the analy-
sis. Also, specific metrics can be chosen carefully to mitigate noisy data. Well-
defined procedures can be used to ensure that appropriate levels of consistency
are achieved in evaluating tasks or usability issues. Many standard question-
naires have already been widely validated by many researchers. The bottom line
is that with some careful thought and a few simple techniques, a lot of the noise
in UX data can be reduced significantly to show a clear picture of user behavior
and attitudes.
Myth 6: You Can Just Trust Your Gut
Many usability decisions are made on a “gut level.” There's always someone on
the project team who proclaims, “This decision just feels right!” One of the beau-
ties of metrics is that having data takes a lot of the guesswork out of usability
decisions. Some design options are truly borderline cases, but they might actually
have an impact on a large population. Sometimes the right design solutions are
counterintuitive. For example, a design team may ensure that all the information
on a web page is above the fold, thereby eliminating the need to scroll. However,
usability data (perhaps in the form of task completion times) may reveal longer
task completion times because there's not enough white space between the vari-
ous visual elements. Intuition is certainly important, but data are better.
Myth 7: Metrics Don't Apply to New Products
Some people shy away from metrics when evaluating a new product. They may
argue that since there is no point of comparison, metrics don't make sense. We
would argue just the opposite. When evaluating a new product, it's critical to estab-
lish a set of baseline metrics against which future design iterations can be compared.
It's the only way to really know if the design is improving or not. In addition, it's
helpful to establish target metrics for new products. Before a product is released, it
should meet basic UX metrics around task success, satisfaction, and efficiency.
Myth 8: No Metrics Exist for the Type of Issues We are
Dealing with
Some people believe that there aren't any metrics related to the particular prod-
uct or project they are working on. Whatever the goal of the project, at least a
couple of metrics should tie directly to the business goals of the project. For
example, some people say they are only interested in the emotional response of
users and not in actual task performance. In this case, several well-established
ways of measuring emotional responses are available. In other situations, some-
one might be concerned only with awareness. Very simple ways to measure
awareness also exist, even without investing in eye-tracking equipment. Some
people say that they are only interested in more subtle reactions of users, such as
their level of frustration. There are ways to measure stress levels without actually