Planning a Usability Study with a Paper Prototype - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 86

cause you to modify the user profile.
Determine the Schedule
A typical first-time usability study using a paper prototype takes a total of 5 or 6 days of effort, but
spread across perhaps a month from start to finish. To recruit users, you have to know when the tests
will be. Finding users has a longer lead time than preparing the prototype, so start by scheduling the
usability tests and allow 3 to 4 weeks for user recruitment. (Once you have a user profile and a
recruitment method established, it may be possible to test on much shorter notice.)
Like they said on Star Trek, "Space: the final frontier." Your testing schedule might depend on when
you can get a place to do it. Try to reserve the same lab/ conference room all day on the days you're
planning to usability test; although paper prototypes are portable, they can be a pain to move.
Work backward from the first test to schedule the activities you'll need to get ready. Although you may
worry that you won't be prepared, chances are you will be—in dozens of paper prototyping projects, I've
never known a team that couldn't get ready in time. One of the advantages of paper is that you can
throw something together at the last minute if need be.
Length of Usability Tests
Perhaps 95% of the usability tests I've done have been either 1 or 2 hours in length. For your first
couple of usability studies, I suggest that you choose between 1-hour or 2-hour tests. Go with 2 hours if
you have many things to cover, if the interface contains new or complex concepts, or if it supports a
work flow that contains more than a handful of steps. For example, in testing software for network
administrators, I've always planned 2-hour tests. On the other hand, 1 hour is usually sufficient if the
interface is smaller in scope.
That's rather vague advice, but fortunately I've found that the product team usually has a pretty good
gut feeling about whether their interface needs 1 hour or 2. Ask yourselves, "Can a user cover what
we're interested in within 1 hour?" and trust your collective instincts. If you're evenly divided, go with 90
minutes.
Of course, tests of other lengths are possible. In my experience, tests longer than 2 hours are rarely
needed—if you plan one, be sure to include time for a break. Tests much shorter than an hour may not
be not practical because you'll want time at the beginning for introductions, time at the end for Q&A,
and you can probably come up with enough tasks to keep the user busy for a while. Short tests might
make sense if you're redesigning only a small part of the interface or you have just one or two tasks;
this is more likely to be the case if you're already conducting frequent usability tests.
Number of Users
As a rule, testing with five to eight users will provide enough data for you to see the main patterns,
provided that you use roughly the same set of tasks (it's okay to vary them a little) and the users are
from the same profile. There is some debate on this number-of-users topic among usability
professionals, and both sides are able to back up their claims with some solid evidence. Nielsen and
Landauer (1993) put forth their finding that the curve describing the total number of problems found
started to flatten out after about five or six users ( Figure 5.1 ). More recently, User Interface Engineering
( Spool & Schroeder, 2001 ), reported that when users are asked to bring their own tasks (and credit
cards) to e-commerce sites, it takes considerably more users to find the major problems—on the order
of dozens. This is because when users do different things, they cover different aspects of the interface.
Not only that, but when users are pursuing goals they truly care about (not to mention spending their
own money), they are more sensitive to subtle issues than when they're doing a task someone else has
invented.