Information Technology Reference
In-Depth Information
probably make many comments without being asked, some negative ( This is
hard or “I don't like this design”) and some positive (“Wow, this is much easier
than I expected” or “I really like the way this looks”). Some comments are neu-
tral or just hard to interpret, such as “This is interesting” or “This is not what I
expected.”
The most meaningful metric related to verbal expressions is the ratio of posi-
tive to negative comments. To do this type of analysis, you first need to catalog
all verbal expressions or comments and then categorize each one as positive,
negative, or neutral. Once this is complete, simply look at the ratio of posi-
tive to negative comments, as illustrated
in Figure 7.1 . Only knowing that positive
comments outnumbered negative com-
ments by a 2:1 ratio does not say a lot
by itself. However, it's much more mean-
ingful if the ratios are compared across
different design iterations or between
different products. For example, if the
ratio of positive to negative comments
has increased significantly with each new
design iteration, this would be one indi-
cation of an improved design. Also, if a
participant is interacting with more than
one design, the same ratio can be cal-
culated for each individual participant,
assuming of course that the time spent
with each product is the same.
100%
80%
60%
40%
20%
0%
Design 1
Design 2
Positive
Neutral
Negative
Figure 7.1 Example of coding the percentage of positive, neutral, and
negative comments for two different designs
It's also possible to get more granular by differentiating among different
types of unprompted verbal comments, such as the following:
Stronglypositivecomments(e.g.,“Thisisterrific!”)
Otherpositivecomments(e.g.,“Thatwasprettygood. )
Stronglynegativecomments(e.g.,“Thiswebsiteisterrible!”)
Other negative comments (e.g., “I don't much like the way that
worked. )
Suggestionsforimprovement(e.g.,“Itwouldhavebeenbetterif… )
Questions(e.g.,“Howdoesthiswork? )
Variationfromexpectation(e.g.,“Thisisn'twhatIwasexpectingtoget.”)
Statedconfusionorlackofunderstanding(e.g.,“Thispagedoesn'tmake
any sense.”)
Statedfrustration(e.g.,“AtthispointI'djustleavethewebsite!”)
These types of data are analyzed by examining the frequency of comments
within each category. Like the previous example, comparing across design iter-
ations or products is the most useful. Categorizing verbal comments beyond
just the positive, negative, or neutral can be challenging. It's helpful to work
Search WWH ::




Custom Search