Performance Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 93

experience metric that doesn't receive as much attention as it should. It's an
essential metric if you need to know how someone develops proficiency with a
product over time.
Consider the following example. Assume you're a user experience profes-
sional who has been asked to evaluate a time-keeping application for employees
within their organization. You could go into the lab and test with 10 partici-
pants, giving each participant a set of core tasks. You might measure task success,
time on task, errors, and even overall satisfaction. Using these metrics will allow
you to get some sense of the usability of the application. Although these metrics
are useful, they can also be misleading. Since the use of a time-keeping applica-
tion is not a one-time event, but happens with some degree of frequency, learn-
ability is very important. A key factor is how much time and effort are required
to become proficient using the time-keeping application. Yes, there may be some
initial obstacles when first using the application, but what really matters is “get-
ting up to speed.” It's quite common in usability studies to only look at a par-
ticipant's initial exposure to something, but sometimes it's more important to
look at the amount of effort needed to become proficient.
Learning can happen over a short period of time or over longer periods of
time. When learning happens over a short period of time, the user tries out dif-
ferent strategies to complete tasks. A short period of time might be several min-
utes, hours, or days. For example, if users have to submit their timesheets every
day using a time-keeping application, they try to quickly develop some type of
mental model of how the application works. Memory is not a big factor in learn-
ability; it is more about adapting strategies to maximize efficiency. Within a few
hours or days, it is hoped that maximum efficiency is achieved.
LEARNABILITY AND “SELF-SERVICE”
Learnability is much more important today than it was in the early days of computers.
The web has facilitated a move toward many more “self-service” applications than we
previously had. At the same time, it has fostered an expectation that you should be
able to use just about anything on the web without extensive training or practice. In
the 1980s, if you wanted to topic an airline flight yourself, you called and spoke to
a representative who had extensive training in the use of a mainframe-based airline
reservation system. Today you go to one of many different websites that let you topic
an airline flight. How long do you think one of those sites would stay in business if it
started by saying, “OK, let's start with a 3-hour training course on the use of our site?”
Learnability is a key differentiator in today's self-service economy.
Learning can also happen over a longer time period, such as weeks, months,
or years. This is the case where significant gaps exist in time between each use. For
example, if you only fill out an expense report every few months, learnability can
be a significant challenge because you may have to relearn the application each
time you use it. In this situation, memory is very important. The more time there
is between experiences with the product, the greater the reliance on memory.