Planning - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 47

Calculating issue severity can help narrow down the cause of specific usabil-
ity problems with a transaction. By assigning a severity to each usability issue,
you will be able to focus on the high-priority problems with any transaction.
Two types of self-reported metrics are also very useful: likelihood to return and
user expectations. In cases where users have a choice of where to perform their
transactions, it's important to know what they thought of their experience. One
of the best ways to learn this is by asking participants whether they would use
the same product again and whether the product met or exceeded their expecta-
tions. Efficiency is an appropriate metric when a user has to complete the same
transaction many times. Efficiency is often measured as task completion per unit
of time.
3.3.2 Comparing Products
It's always useful to know how your product compares to the competition or to
previous releases. By making comparisons, you can determine your product's
strengths and weaknesses and whether improvements have been made from one
release to another. The best way to compare different products or releases is
through the use of various metrics. The type of metrics you choose should be
based on the product itself. Some products aim to maximize efficiency, whereas
others try to create an exceptional user experience.
For most types of products, we recommend three general classes of metrics to
get an overall sense of the user experience. First, we recommend looking at some
task success measures. Being able to complete a task correctly is essential for
most products. It's also important to pay attention to efficiency. Efficiency might
be task completion time, number of page views (in the case of some websites),
or number of action steps taken. By looking at efficiency, you will get a good
sense of how much effort is required to use the product. Some self-reported
metrics of satisfaction provide a good summary of the user's overall experience.
Satisfaction measures make the most sense with products where people have
choices. Finally, one of the best ways to compare the user experience across
products is by combined and comparative metrics. This will give an excellent
big picture of how the products compare from a UX perspective.
3.3.3 Evaluating Frequent Use of the Same Product
Many products are intended to be used on a frequent or semifrequent basis.
Examples might include microwave ovens, mobile phones, web applications
used as part of your job, and even the software program we used to write this
topic. These products need to be both easy to use and highly efficient. The
amount of effort required to send a text message or download an application
needs to be kept to a minimum. Most of us have very little time or patience for
products that are difficult and inefficient to use.
The first metric we would recommend is task time. Measuring the amount
of time required to complete a set of tasks will reveal the effort involved. For
most products, the faster the completion time, the better. Because some tasks