Planning - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 48

are naturally more complicated than others, it may be helpful to compare task
completion times to expert performance. Other efficiency metrics, such as the
number of steps or page views (in the case of some websites), can also be help-
ful. The time for each step may be short, but the separate decisions that must be
made to accomplish a task can be numerous.
Learnability metrics assess how much time or effort is required to achieve
maximum efficiency. Learnability can take the form of any of the previous effi-
ciency metrics examined over time. In some situations, consider self-reported
metrics, such as awareness and usefulness. By examining the difference between
users' awareness and perceived usefulness, you will be able to identify aspects
of the product that should be promoted or highlighted. For example, users may
have low awareness for some parts of the product, but once they use it, they find
out it is extremely useful.
3.3.4 Evaluating Navigation and/or Information Architecture
Many usability studies focus on improving the navigation and/or informa-
tion architecture. This is probably most common for websites, software pro-
grams, mobile applications, consumer electronics, interactive voice responses,
or devices. It may involve making sure that users can find what they are look-
ing for quickly and easily, navigate around the product easily, know where they
are within the overall structure, and know what options are available to them.
Typically, these studies involve the use of wireframes or partially functional pro-
totypes because the navigation and information mechanisms and information
architecture are so fundamental to the design that they have to be figured out
before almost anything else.
One of the best metrics to evaluate navigation is task success. By giving par-
ticipants tasks to find key pieces of information (a ''scavenger hunt''), you can
tell how well the navigation and information architecture works for them. Tasks
should touch on all the different areas of the product. An efficiency metric that's
useful for evaluating navigation and information architecture is lostness, which
looks at the number of steps the participant took to complete a task (e.g., web
page visits) relative to the minimum number to complete the task.
Card sorting is a particularly useful method to understand how participants
organize information. One type of card-sorting study is called a closed sort,
which has participants put items into predefined categories. A useful metric to
come from a closed card sort study is the percentage of items placed into the
correct category. This metric indicates the intuitiveness of the information archi-
tecture. There are some helpful online tools to collect and analyze this type of
data, such as Optimal Sort and Treejack (developed by Optimal Workshop in
New Zealand).
3.3.5 Increasing Awareness
Not every design that goes through a usability evaluation is about making some-
thing easier or more efficient to use. Some design changes are aimed at increasing