Database Reference
In-Depth Information
each one is an issue, a bug, or an anomaly. Duplicates are discarded. At the end of this process
we have one stack of stickies that's much smaller than the piles we started with. When an issue
comes up and it's clear who should solve it, that person gets the sticky note. Sometimes
everyone walks away with their set of things to solve, and there's nothing to write up afterward.
Other times, we take the single stack of issues and prioritize those by an affinity diagram. The
prioritized issues go into a spreadsheet—in a later meeting we'll read through the issues and
assign them. This variation is helpful when a lot of data is obtained because it postpones the
discussion of how to solve the issues until priorities are agreed upon."
Mary Beth Rettger, The MathWorks
Why a Group Method?
You may wonder why you'd want to do the prioritizing process as a group, especially if there are only a
couple of people who'll be implementing the changes. Here are some reasons to consider.
Pooling observations. No one person can take complete and perfect notes. Because each
observer captures a different subset of observations, there's less chance that an issue will slip
through the cracks.
Recognizing patterns. Most usability test observations are qualitative in nature—they're not
something you can measure with a yardstick or a voltmeter. The key to analyzing qualitative data is
to identify the patterns over several users' worth of tests. Although I don't know much about pattern
recognition as a science, I do know that it's a skill and some people are better at it than others.
Similar to how a group stands a better chance of wilderness survival than any of its individual
members, having several people look at the big picture usually means a greater chance of correctly
identifying the most important issues.
Choosing optimal solutions. It's easy to get focused on your particular area of responsibility and
lose sight of the larger picture. When this happens, problems may not get solved in the most
effective way. For example, a tech writer's first reaction to an inscrutable error message might be,
"Gee, I'd better document that." (Or perhaps, "Let me improve the wording of that error message.")
But a developer sitting in the same room might suddenly realize, "You know, I could put a check in
the code to prevent that problem from happening at all." Conversely, the team might decide that
documenting an issue really is the only feasible way to address it. By having the team discuss the
options, there's a better chance of finding an efficient solution. An additional benefit is that the left
hand will know what the right hand is doing—if the developer says that the error message will be
eliminated, the tech writer won't waste time documenting it.
Disseminating results. The more members of the product team who attend the debriefing, the
less need there may be for formal documentation of the results via reports, highlight tapes, or other
time-consuming activities.
A Note about Granularity
Some people record their issues directly onto the index cards or sticky notes while they observe the
usability tests. I think this is okay, but be aware that it can have an effect on the granularity of the
observations you collect. Not all usability issues occur at the level of a confusing field label—some
unfold as a series of events over several minutes or longer. For example, perhaps you initially thought
that the users made a mistake but later realize that they had a different and valid way of going about
the task. I find that I sometimes have to look over a few pages of my notes to piece together an issue
that is subtle and/or happened in the context of several screens. If you use stickies to record your
notes, it might be a good idea to spread out all your stickies in order, see if any larger issues jump out
at you, and if so then summarize them on stickies of their own.
Search WWH ::




Custom Search