Observers - Paper Prototyping: The Fast and Easy Way to Define and Refine User Interfaces - page 167

directly, especially when the facilitator is not a subject matter expert but the users are. Although it's
possible for the facilitator to run next door to an observation room (or use a phone or other equipment)
to gather questions from observers, I've found that observers in these situations ask fewer questions
than when they're in the same room. (Note that it is not appropriate for observers to interrupt a task with
questions. As mentioned in Chapter 8 the facilitator decides when questions are allowed and how long
discussions should continue.)
The Facilitator Can Observe the Observers
Effective usability test facilitation contains a feedback loop: Users provide data to developers;
developers respond by taking notes, through body language, and by asking questions; and the
facilitator makes adjustments to help the observers get the most out of the session. For example, I
might see that all the observers are busily writing notes. This lets me know that they've noticed an
important issue, so I might slow down the pace until they've captured it. (Plus, I don't have to harp on
that issue later—I know they've gotten the message.) On the other hand, if observers are sitting there
passively, either they already have ample data on the issue or the current line of investigation isn't of
interest. Knowing that, I can respond by assisting the user, moving to a different task, or changing the
topic of discussion.
My effectiveness as a facilitator is hampered if I can't see how the observers are reacting—I'm dealing
with an open feedback loop. My worst example was when I went next door to the observation room at
the end of a test to see if the observers had any questions, and instead of the half-dozen people I was
expecting, there was no one there. I had no idea when or why they had left, what they'd seen, or if
anyone had bothered to take notes. I felt like that lonely tree that fell in the forest.
There Are No Hidden Watchers
Although you might assume that it's more intimidating to have observers present, some users have told
me the opposite—the thought of hidden watchers can be unnerving. At the time of this writing, many
companies are clarifying their policies regarding the use of personal information collected from
customers. In the future, as people become more savvy about their rights to safeguard their information
and privacy, it is possible that unseen observers and videotaping may become less acceptable to
people than they are today. On the other hand, if video surveillance of public places becomes more
common, people may come to think of their lives as one big reality TV show. Either way, the social and
legal issues around this question bear watching (no pun intended).