Observers - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 175

agree that these are the ideals we're trying to uphold." Another tactic for raising awareness is to ask the
observers to brainstorm what might happen if someone violated each of the rules.
Observers Are People Too
Right before a usability test, a developer once joked with me that he wished he had a cast iron lining in
his stomach because he was so nervous about what the test would reveal. This from a guy who
wholeheartedly supported the idea of usability testing, and after the first couple of tests had found only
minor problems! With all the focus on users, it's easy to lose sight of the fact that observers have
feelings too.
Usability tests can be nerve-wracking, especially to those who had a direct hand in designing the
interface. So I end my observer briefing by acknowledging that a usability test can be stressful and
frustrating for observers as well as for users. As a facilitator, my job isn't to beat up the interface (and,
by proxy, the people who designed it) but to shed light on what's happening inside the users' heads. I
emphasize that part of our purpose is to find the parts of the prototype that are working well, not just
the flaws. This is something that's easy for beginning facilitators to forget in their enthusiasm for
uncovering problems.
In June 2001 Bruce Tognazzini wrote a brilliant, biting, and all-too-true column on this subject
called "How to Write a Report without Getting Lynched." It's on his Web site at
www.asktog.com . His opening sentence is, "The finest set of recommendations will be rejected
if the form in which they are received is seen as hostile or belligerent." This article should be
required reading for anyone who conducts usability tests or reports their results.
When to Brief Observers
For several years, I made it a point to brief all observers right before every test, just to be on the safe
side. Although I no longer think this is necessary, I might still do this if there were individuals I was
concerned about but didn't want to single out. What I usually do now is designate one or two 10-minute
briefings on the first test day—maybe one first thing in the morning and one after lunch—and tell
observers that they must attend one of them before coming to a test. If someone can't attend a briefing,
I'll either arrange a special briefing for that person or I'll "deputize" a trusted observer to do so.
Relaxing the Rules
When I'm not busy conducting usability tests, one of my hobbies is motorcycling. I've found that the
best passengers are those who are just a little bit scared because they will do exactly what I tell them.
But if they get too relaxed back there, they're more likely to do something dangerous. The worst
passenger I ever had was a proficient rider who was so comfortable that he kept forgetting his actions
affected my control of the bike—like the time he leaned over to scratch his ankle and almost dumped
us both onto the pavement.
I've seen a similar problem with usability test observers. Most observers are initially conscientious
about their behavior, especially after hearing the story of The User Who Cried, and they follow the rules
to the letter. But sooner or later the observers become comfortable with the test format, and they start
to bend the rules. It starts in fairly innocuous ways, such as interrupting a task with, "Can I ask just one
question?" (It's difficult to say no because the observer genuinely wants to know, and I've never been
able to bring myself to use the flippant response, "Yes, and you just did.")
But this is a slippery slope, and it can affect the facilitator's control of the test session. One or two
interruptions may not spell disaster, but they open the door to similar behavior that eventually might.
The worst case I experienced was when observers managed to start separate conversations with the
two users, and the test fragmented into a rudderless meeting. After that incident, I carefully rebriefed
the team again, telling them, "Even though we're quite comfortable with how a usability test works, we
have to remember that it's all new to the users." This line is a good reminder to experienced observers
that their behavior can have consequences they didn't intend.
Having said all this, sometimes no harm comes from relaxing the rules a bit, especially if there are only