Database Reference
In-Depth Information
Debriefing Meeting: Prioritizing the Issues
Usability testing is like getting a drink from a fire hydrant—there's a whole bunch of data, coming at you
at high speed. So what do you do with it all? It's important to get the product team together at the end
of a usability study and sort out what you learned. Even if you've been listing issues and making
changes after each test, it's still useful to step back and look at the larger picture. This happens at the
debriefing meeting.
There are two main outputs from the debriefing process: prioritization of the issues and an action plan
for addressing them. Some teams do both things at one meeting; if some observers aren't involved in
implementing changes, they can participate in the prioritization but leave when you're ready to create
the action plan.
Because it's usually not practical for everyone to observe every test, people may arrive at the debriefing
meeting with different ideas about what happened. So the first order of business is to share everyone's
observations and agree on priorities. Then and only then should you start discussing what to do about
the issues.
Affinity Diagram
A technique called an
affinity diagram
is useful for identifying the patterns in qualitative data.
Figure
11.1
shows what one looks like, and
Figure 11.2
describes the process (a handout is available at
www.paperprototyping.com
). Here's a brief overview: Everyone picks their top observations from their
notes and writes them on individual sticky notes or blank index cards. (At this point I remind people that
their inferences and opinions are best saved for later, although inevitably a few of them will creep in.)
Tape all the cards to a wall. Everyone reads all the observations, and then you sort them into groups,
name each group, and then vote on which group(s) have the greatest impact on the success of the
next release.
Figure 11.1:
In an affinity diagram, the team sorts individual observations into related groups.
Putting flip chart paper on the walls first as shown here makes the whole thing easily portable to
another wall.
1.
Observers go through their notes and identify the unsolved issues that they believe
are most important to the success of the next release. They write those issues,
one
per card,
onto index cards (or sticky notes). You may want to have a rule that the
number of tests affects the number of cards from each person—perhaps about 5 per
test.
2.
Tape all the cards to one wall in random order.
3.
Everyone reads all the cards. Don't worry about duplicates or issues that were solved
by subsequent prototype changes—keep those issues in the process. (Variation: A
Search WWH ::
Custom Search