Database Reference
In-Depth Information
causing dissent among the product team because it's unclear to what extent the test results are
credible. Worst case, the team may simply dismiss the findings (perhaps correctly so) as irrelevant.
Bias: Bad Tasks
Because I believe that task design is one of the most important and difficult aspects of usability testing,
I already devoted all of
Chapter 6
to this topic. Tasks can be a subtle and insidious source of bias. As
I've learned from my own mistakes, it's easy to give unintended hints that can conceal usability
problems in the interface. On the other hand, unrealistic tasks can cause users difficulty, but that
difficulty may not be indicative of what they would experience in real-life use.
Sometimes the clues contained in task instructions are blatant (at least in retrospect), and other times
they can be subtle.
Figure 13.1
shows a search form on a travel Web site. In testing this site, we used
the following task: "You and your significant other are planning a visit to Madrid, Spain. You're arriving
September 9th and staying for 7 nights, departing the morning of the 16th. A friend recommended the
hotel Husa Princesa. Please use the Web site to topic a room." (The purpose of this task was to find
out how a user would react to the error message telling them that one or more of the requested nights
was unavailable.) As
Table 13.1
shows, there are several subtle biases lurking in even this simple task.
Table 13.1: Possible Sources of Bias in Task Instructions
Possible Bias
Discussion
The instructions reveal that
Madrid is in Spain.
We did this deliberately—the team assumed users would
already have done some research about their destination and
we didn't want to waste test time if they didn't happen to know
where Madrid was. But otherwise, this bit of information might
have masked a problem with the country being required, or
possibly users' need to see a map.
The instructions give the
spelling of the hotel name.
Also deliberate—the team already understood the problems
caused by spelling variations, and we wanted to ensure that
they'd use this particular hotel because it lacked available
rooms on the dates in question.
The task assumes that the
user has a certain set of
information when they use
the site, such as exact
travel dates.
An important issue—if the search requires users to enter
information they don't have, they'll be stuck. But if we give
them all the inputs, we won't see this problem. (In this study,
we had an earlier task where we asked users to find a hotel for
a vacation they were planning in real life, so we'd already
gotten data about this, including the fact that people didn't
always have exact dates.)
The order of the information
in the task was different
than the order of the fields
in the search form—the
dates and hotel name were
reversed.
We didn't realize this. One user made an error because she
followed the order in the task instructions and submitted the
form after entering the dates. (Only a minor issue in this case,
but it could have been important if we were testing the search
form.)
Search WWH ::
Custom Search