When to Use Paper - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 246

Tasks, Data, and Test Scenarios
The usability tasks that you want users to do can have a bearing on whether paper will offer
advantages over testing the real thing. Paper prototypes can sometimes give you greater flexibility with
your tasks than you'd get with the real interface. In other cases they're a poor substitute for reality.
Control of Content
You don't just need control of the technology—for usability testing purposes, you often need some
degree of control over what the user sees, such as information that comes from a database. For live
Web sites, don't forget content such as ads that may change without warning (not to mention the risk of
accidental pornography). If some of the content isn't under your control, think about whether this could
have a detrimental effect on your usability tests.
Between-Test Reset Procedures
In setting up a series of usability tests, consider what you need to do between tests to reset the test
environment. For a Web site, this may be as simple as clearing the link history in the browser and
deleting cookies. For software applications, it may be easy enough to delete all the files the user
modified during the test session and replace them with fresh copies. But not always. One Web
application had an underlying work flow involving several users and databases—when Person A was
done handling a request, it resulted in database changes and work being sent to Person B via email.
To simulate the work flows needed for our tasks, one of the developers created some elaborate
procedures that we had to run between tests, and in one case even between tasks. (And because he
couldn't attend all the tests, he also had to train others on those procedures.) I asked him afterward to
estimate how long he'd spent on all that setting up and resetting; he said about 15 hours. Because this
interface was fairly simple, we could have created a paper prototype in perhaps half that time.
Installation and Configuration Tasks
Tasks involving installation or configuration are a special case of reset procedures—they can be a pain.
Say you have a test scenario where the user installs a software application. Perhaps in task 1 you'd
test the ideal case where everything goes smoothly, and in task 2 you want to throw the user a curve in
the form of a compatibility problem they have to resolve. To do task 2 , you'd have to uninstall the
software and set up the conditions that would cause the incompatibility. Or you could save an image of
the pristine test machine on a server and restore it, although in my experience this can result in several
minutes of downtime (which may not be a drawback if you can hold a discussion with the users in the
meantime). With a paper prototype, you could simply start task 2 by telling the user, "Assume you're
doing the same thing on a different machine."
Real-World Consequences
I once visited a prospective client that was developing a Web application for booking corporate travel.
They knew that their interface lacked feedback, and rather shamefacedly admitted that on a couple of
occasions while testing the interface they had accidentally bought plane tickets ... nonrefundable ones,
naturally. That's an example of a real-world consequence. Similarly, some software applications control
physical equipment. If you're testing an interface for medical equipment or some other potentially
dangerous device, you may have a situation in which users can blithely get themselves into real-world
trouble. I once taught paper prototyping to some engineers who worked on process control for heavy
machinery used in processing steel. One of them remarked that you got a whole new appreciation for
the effects of a bug when it could result in sheets of steel spewing across the factory floor at 200 mph.
Obviously, paper prototypes avoid such problems.
Lengthy Processing Delays
It's not a big deal in a usability test if the user occasionally has to wait a few seconds for a page to
download or a large file to be opened. But if there is processing that takes any longer than the time