Database Reference
In-Depth Information
Inferences
are conclusions based on some observations and possibly some assumptions as well.
Examples of inferences are "User is lost" or "This dialog box is confusing." Unlike observations,
inferences are not fact; two observers can and do draw opposite inferences from exactly the same
observations. As a way of checking whether something is an observation or an inference, ask yourself,
"How do I know that?" If it could be verified from a videotape exactly as you've described it, it's probably
an observation. But if it contains an assumption about what was going on inside the user's head, it is
probably an inference. For example, the statement "He likes this version better" is an inference unless
the user actually
said,
"I like this version better," in which case then it is indeed an observation.
An
opinion
is something you think should be done about your inferences or observations, such as,
"Search results should by sorted by increasing price" or "We need to show an example." (The words
should
and
need
are often clues that you've written an opinion in your notes.) Ideas that occur to you
during usability tests are not necessarily bad, but you don't want to confuse them with things that users
have asked for.
Note
Sometimes I do deliberately include inferences or opinions in my notes, but I flag them (for
example, with "???" or "idea") so that I know these are just my thoughts, not necessarily
reality.
If you're not aware of these distinctions, especially between observations and inferences, it's easy to
confuse them when looking back over your notes. The problem with inferences and opinions is that two
observers can see the same thing but put a different spin on it. Here's an example of several possible
inferences that could be drawn from one observation:
Observation:
User paused for several seconds before entering "My IRA" in the Description field for
the account.
Inference 1:
He didn't know whether the description was required.
Inference 2:
He wasn't sure whether spaces were okay.
Inference 3:
He wondered whether this description would appear in his portfolio instead of the
account name.
Inference 4:
He wasn't confused; he was just deciding what to call it.
And so on. As you can see, the people making these inferences have drawn very different conclusions
about what the problem is (or whether there is one) and will go on to formulate different opinions about
how to solve it. Thus, I think of inferences and opinions as "argument seeds" planted in one's notes,
because they're likely to sprout into disagreement later.
A good test facilitator encourages users to articulate what they're thinking and doing, thus turning what
would otherwise be inferences into observations. (This is the essence of the sportscaster role.) For
example, if the user paused for more than a few seconds, the facilitator might ask, "What's going
through your mind right now?" The user's answer should reveal which of these situations is happening.
Inferences and opinions in your notes are "argument seeds," likely to sprout into disagreement
later.
At some point you probably do want to start determining the possible causes of problems, but it's best
to wait until the debriefing meeting to do this. Let's assume that everyone wrote down the observation
that the user paused. Some possible reasons for this might be supported by additional observations;
for example, if the user later said, "Good, it took it" then that observation might support inference 2
about whether the description could include spaces. Sometimes there will still be disagreements, but if
everyone offers their inferences as fact, it becomes much harder to figure out what really happened.
I'm not advocating that we should overanalyze everything to ridiculous lengths. I deliberately chose a
rather trivial example of a momentary pause to illustrate the point that some problems are more
important than others. As I explain later in this chapter, by first prioritizing the issues that were found in
the usability tests, the team will avoid dissecting the minor ones.
Overanalyzing details to ridiculous lengths reminds me of the joke about a scientist who takes a
Search WWH ::
Custom Search