Planning - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 56

Lab first, then online Online first, then lab

Identify/fix “low-hanging fruit”and
then focus on remaining tasks with
large sample si z e

Identify the most significant issues
online through metrics and then
use lab study to gather deeper
qualitative understanding of those
issues

Generate new concepts, ideas, or
questions through lab testing and
then test/validate online

Collect video clips or more quotes
of users to help bring metrics to
life

Validate attitudes/preferences
observed in lab testing

Gather all the metrics to validate
design—if it tests well, then no
need to bring users into the lab

3.4.3 Online Surveys
Many UX researchers think of online surveys strictly for collecting data about prefer-
ences and attitudes, and firmly in the camp of market researchers. This is no longer
the case. For example, many online survey tools allow you to include images, such as
a prototype design, within the body of the survey. Including images within a survey
will allow you to collect feedback on visual appeal, page layout, perceived ease of use,
and likelihood to use, to name just a few metrics. We have found online surveys to be
a quick and easy way to compare different types of visual designs, measure satisfac-
tion with different web pages, and even preferences for various types of navigation
schemes. As long as you don't require your participants to interact with the product
directly, an online survey may suit your needs.

The main drawback of online surveys is that the data received from each par-
ticipant are somewhat limited, but that may be offset by the larger number of
participants. So, depending on your goals, an online survey tool may be a viable
option.

INTERACTING WITH DESIGNS IN AN ONLINE SURVEY

Some online survey tools let participants have some level of interaction with images.
This is exciting because it means you can ask participants to click on different areas
of a design that are most (or least) useful or where they would go to perform certain
tasks. Figure 3.3 is an example of a click map generated from an online survey. It shows
different places where participants clicked to begin a task. In addition to collecting data
on images, you can also control the time images are displayed. This is very helpful in
gathering first impressions of a design or testing whether they see certain visual elements
(sometime referred to as a “blink test”).