Introduction - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 8

standard metrics such as the Presidential Approval Ratings, the Consumer Price
Index, or the frequency of purchasing specific products. Although these metrics
are all quantifiable and may reflect some type of behavior, they are not based on
actually using something in order to reflect the variability in the data.

UX metrics are not an end unto themselves; rather, they are a means to help
you reach an informed decision. UX metrics provide answers to questions that
are critical to your organization and that can't be answered by other means. For
example, UX metrics can answer these critical questions:

• Willtheusersrecommendtheproduct?

• Isthisnewproductmoreefficienttousethanthecurrentproduct?

• Howdoestheuserexperienceofthisproductcomparetothecompetition?

• Dotheusersfeelgoodabouttheproductorthemselvesafterusingit?

• Whatarethemostsignificantusabilityproblemswiththisproduct?

• Areimprovementsbeingmadefromonedesigniterationtothenext?

1.3 THE VALUE OF UX METRICS
We think UX metrics are pretty amazing. Measuring the user experience offers
so much more than just simple observation. Metrics add structure to the design
and evaluation process, give insight into the findings, and provide informa-
tion to the decision makers. Without the insight provided by metrics, important
business decisions may be made based on incorrect assumptions, “gut feelings,”
or hunches. As a result, some of these decisions are not the best ones.

During a typical usability evaluation, it's fairly easy to spot some of the
more obvious usability issues. But it's much harder to estimate the size or
magnitude of the issues. For example, if all eight participants in a study have
the same exact problem, you can be quite certain it is a common problem.
But what if only two or three of the eight participants encounter the prob-
lem? What does that mean for the larger population of users? UX metrics
offer a way to estimate the number of users likely to experience this prob-
lem. Knowing the magnitude of the problem could mean the difference
between delaying a major product launch and simply adding an additional
item to the bug list with a low priority. Without UX metrics, the magnitude
of the problem is just a guess.

User experience metrics show whether you're actually improving the user
experience from one product to the next. An astute manager will want to know
as close to certain as possible that the new product will actually be better than
the current product. UX metrics are the only way to really know if the desired
improvements have been realized. By measuring and comparing the current
with a new, “improved” product and evaluating the potential improvement, you
create a win-win situation. There are three possible outcomes:

• Thenewversiontestsbetterthanthecurrentproduct:Everyonecansleep

well at night knowing that improvements were made.