When to Use Paper - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 244

Development Context
As you can tell from the war stories, a lot of things can prevent you from conducting usability tests. The
factors discussed in this section will help you estimate the likelihood of a technical snafu interfering with
your testing plans.
Dependence on Technology
Prerelease stuff is buggy. As Murphy's Law says, anything that can go wrong, will go wrong. This state
of affairs is normal during product development, but it can wreak havoc with your usability testing plans.
The more modules or applications that have to play nicely together, the more vulnerable you are to
technical problems—and the effect is multiplicative. Say you have a task that utilizes four software
modules. Each performs reliably 90% of the time and crashes the other 10%. Put these four modules
together, and your chance of getting through the task without crashing is 0.9 × 0.9 × 0.9 × 0.9, or only
about 66%. It's kind of like Murphy's Law with multiple Murphys. In contrast, paper prototypes don't rely
on databases, networks, servers, or any other form of technology, which eliminates most of the things
that can go wrong. I've even done paper prototyping during a power failure!
You programmers out there, I know what you're thinking: "But this is my application. I'm the one writing
the code." Yes, but until you're well into the debugging stage, your code (or code written by others that
yours depends on) may not be stable enough to allow users to do meaningful work without crashing.
Bugs happen, and software development is sometimes a process of taking two steps forward and one
back.
It is possible to conduct usability tests with buggy software, and I have done this many times. If the
development team knows where the land mines are located, the facilitator can usually steer users
safely around them. If the users find a land mine we didn't know about, perhaps it's not difficult to
restart the system and get back to where they left off. But these situations can be nerve-wracking, not
only for the users but also for the product team. As a facilitator, I sometimes find myself mentally
chanting, "Stop him before he clicks Submit, stop him before..." Which is not so bad—it's part of the
facilitator's job—but the real issue is that the observers have the same chant going through their heads.
I've seen developers who were so focused on the survival of their interface that they didn't get much
value from watching the usability tests. If you're concerned that a buggy interface will be too distracting
to the product team (or upsetting to users, who may disregard assurances that it isn't their fault),
consider using a paper prototype instead.
Usability Testing versus Development
I deliberately put the "versus" in the heading because sometimes that's how it seems to me—it's
possible for usability testing to interfere with the development process and vice versa, especially in its
later stages. The degree to which a project is vulnerable to this depends on whether there's a separate
environment that can be used for testing. If the usability tests rely on the latest build on the
development server, development must come to a carefully orchestrated pause so that someone