Task Design - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 104

Step 5: Number and Order the Tasks
So how many tasks should you create? Enough to fill your allotted test time, plus one or two extra.
Keep in mind that you won't have all the test time to spend on the tasks—it typically takes 5 to 10
minutes to get the user comfortable, briefed, and introduced, perhaps longer if you interview the user
about his or her background or work. You may want to reserve another 10 minutes at the end of the
test for questionnaires or to discuss interesting issues that arose. So the time available for tasks is
typically the test length minus 10 to 20 minutes.
The Time Expansion Factor
Estimating task times is an art, not a science. Although I've gotten better at it over the years,
occasionally I'm still way off. But there's a rule of thumb that I've found useful: Take that "time for
expert" estimate and multiply it by a time expansion factor. This factor is a crude means of accounting
for all the things that cause people in a usability test to work more slowly than an expert under ideal
conditions: they're unfamiliar with the interface, they're verbalizing their thought process, they might
have to wait for the Computer, and so on.
For software, I multiply the expert's time by a factor or 5 to 10 (I usually use a range, not a single
number). In other words, a task that takes an expert 1 minute under ideal conditions will probably
consume 5 to 10 minutes of test time. For Web sites, I've found that the time expansion factor seems
to be smaller, more like 3 to 5 minutes. (I think this may be because the controls are usually simpler
than for software—users are primarily clicking—but this is just a guess.) In looking at your interface,
think about how much data entry is involved. Writing by hand takes time, so lots of data entry implies a
bigger expansion factor. Also consider how many new or complex concepts are involved, which will
affect the amount of time users need to think and/or experiment. Of course, these numbers are only
rough guidelines, and your mileage will vary.
Add up the expanded task times and see whether you've filled your allotted test time. It's a good idea to
create an extra task or two. Especially in a paper prototype, it's common for users in later tests to
complete more tasks because your improvements to the interface make it easier to use.
Make sure that your tasks aren't too long. I've found that a task longer than 30 or 40 minutes can be
fatiguing for users (not to mention observers). But there's no hard and fast definition of "too long." If
you're testing a lengthy process, you might want to break it down into smaller tasks. Or the facilitator
can plan to intervene after a certain amount of time, ask the users to summarize what they've done so
far, and offer them a break.
Also be on the lookout for tasks that are very short—5 minutes or less. Sometimes this indicates that
you're testing just one screen or function rather than a user goal. Although a short task is not
necessarily bad, ask yourself if it makes sense to combine it with another task.
Dropping and Reordering Tasks
For some types of usability testing it's important to use the same tasks in the same order for each user.
This is typically the case if the purpose of the testing is to collect success rates, task times, error rates,
or other metrics. But metrics usually aren't the focus of paper prototype tests because the interface is
continually evolving. Thus, it's usually not important to use the same tasks in the same order for each
usability test—it's okay to vary them.
The purpose of paper prototyping is to provide answers to the questions you're most concerned about.
As you conduct usability tests, you'll collect lots of data, and this data can cause you to be more (or
less) worried about a particular issue. If you find your concerns changing during a usability study, it's
appropriate to change the tasks accordingly. Here are some reasons why you might decide to drop a
task and replace it with one of the extras you created:
You're not learning anything new. Sometimes you learn all you need from a task by watching
just the first few users complete it, especially if the interface works well (which does happen,
although not as often as we might like). Once a task stops yielding new insights, it's done its job