Planning - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 55

• Card-sorting/IA the

researcher collect data about how
users think about and organize infor-
mation. Tools such as OptimalSort,
TreeJack, and WebSort are very use-
ful, easy to set up, and affordable.

tools help

• Surveysareincreasinglybecomeuse-

ful to the UX researcher. Tools such
as Qualtrics, SurveyGizmo, and
SurveyMonkey let the researcher
embed images into the survey and
collect a wide variety of self-reported
metrics, along with other useful click
metrics.

• Click/Mouse as

Chalkmark, Usabilla, ClickTale, and
FiveSecondTest let the researcher
click data about where users click on
a web page or how they move their mouse around. These tools are useful
for testing the awareness of key features, intuitiveness of the navigation,
or what just grab users' attention the most.

tools such

Figure 3.2 A breakdown of the different types of online (unmoderated)
testing tools.

Qualitative-based online tools are designed to collect data from a small num-
ber of participants who are interacting with a product. These tools are extremely
helpful for gaining insight into the nature of the problems that users encoun-
ter, as well as provide direction on possible design solutions. There are different
types of qualitative-based tools.

• VideotoolssuchasUserTesting.com,Userlytics,andOpenHallwayallow

you to collect a rich set of qualitative data about the users' experience in
using a product in the form of a video file. Observing these videos lets
the researcher collect performance metrics, and possibly self-reported
metrics, depending on the capabilities of the tool.

• Reportingtoolsprovidetheuserwithanactualreportthatistypicallya

list of verbatim comments from users about their experience in using the
product. The metrics may be limited, but it is certainly possible to do text
analysis of the feedback, looking for common trends or patterns in data.

• ExpertreviewtoolssuchasConceptFeedbackprovidetheuserresearcher

with feedback from a group of “experts” about a product's design and
usability. While the feedback is typically qualitative in nature, the
researcher might also collect self-reported metrics from each reviewer.

WHICH ONE GOES FIRST? LAB OR ONLINE TEST?

We often get questions about which should go first, a traditional lab study, followed by
an online study, or vice versa. There are some pretty strong arguments for both sides.