Observers - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 174

opinions about how to design aspects of the application (such as, "Where would you like to see
these navigation buttons?") can take a lot of time to answer and produce only limited results.
Instead, focus on trying to understand the problem —we'll come up with solutions later, outside the
test.
Respect Participants and the Confidentiality of Their Data
We have promised the participants that their participation is confidential. This means that we
should not include their names in any reports or other communication such as email, and we
should refrain from discussing them by name outside the test setting. Do not make negative
comments about people—there is always a risk that a derogatory comment could be overheard or
otherwise make its way back to the user.
How to Explain the Rules to Observers
I suppose I could just hand observers the Rules and say, "Do this or else!" But people are more likely
to adhere to the spirit of the rules when they understand their purpose and importance. So I start out
very seriously. A good way to drive home the purpose of the rules is to describe a worst-case scenario,
like the one that happened to Jared Spool (see the From the Field box on p. 232). Empathizing with the
user's shame and embarrassment creates a "teachable moment" where people are much more inclined
to think about the consequences of their behavior.
From the Field: The User Who Cried
"I once had a user cry during a usability test. This was years ago, before I had much experience
with usability testing. Basically, it happened because we'd done just about everything wrong.
When our scheduled user didn't show up, someone grabbed an employee to fill in, without
explaining what the session was about—that was the first mistake. Turns out it was this woman's
first day on the job, and we figured she'd be a good candidate because she knew absolutely
nothing about the product. Not only did she know nothing about the product, but it wasn't even in
her area of expertise—she didn't fit the user profile at all. That was the second mistake. There
were a bunch of observers sitting in the room who hadn't been briefed on how to behave (third
mistake), including the woman's manager (fourth mistake). And last but not least, we hadn't done
a rehearsal beforehand, so we didn't realize that the first task we wanted the user to do was, in
fact, impossible.
"Once the user started running into difficulty, everyone except for her quickly realized that the
task was ridiculous, and they all started laughing at their own stupidity. Unfortunately, the user
thought they were laughing at her and she started crying. We all felt horrible, and after that I took
the time to hand out a written set of rules and go over them with every observer before they're
allowed to sit in the room. Thankfully, nothing like this has ever happened to me again."
Jared M. Spool, User Interface Engineering
( www.uie.com )
I don't usually find it necessary to discuss each rule in detail, but there is no harm in doing so,
especially when usability testing is new to a company. (I usually do remind people that when I ask a
question, I'm asking the users. Otherwise, it's common for a helpful team member to pipe up with the
answer.)
As an outside consultant I have the advantage (deserved or not) of coming into a company as the de
facto expert in usability, and as a result I find that observers are willing to follow my instructions. But if
you are self-taught in usability and everyone on your team knows it, that kind of credibility—and the
authority that accompanies it—may be a bit harder to come by. One excellent suggestion from veteran
usability specialist Chauncey Wilson is to have observers sign a copy of the rules once they've read
and understood them. Chauncey notes, "The commitment of a signature seems to work, even with
senior managers." To put it another way, it turns the situation from "You must do as I tell you," into "I