Special Topics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 222

Cards that join together sooner are more similar to each other than those that
join together later. For example, the pair of fruits with the lowest (shortest) dis-
tance in Table 9.8 (peaches and oranges; distance = 2) join together first in the
tree diagram.
Several different algorithms can be used in hierarchical cluster analysis to deter-
mine how the “linkages” are created. Most of the commercial packages that sup-
port hierarchical cluster analysis let you choose which method to use. The linkage
method we think works best is one called the Group Average method. But you
might want to experiment with some of the other linkage methods to see what the
results look like; there's no absolute rule saying one is better than another.
One thing that makes hierarchical cluster analysis so appealing for use in the
analysis of card-sorting data is that you can use it to directly inform how you
might organize the cards (pages) in a website. One way to do this is to take a
vertical “slice” through the tree diagram and see what groupings that creates. For
example, Figure 9.4 shows a four-cluster “slice”: The vertical line intersects four
horizontal lines, forming the four groups whose members are color coded. How
do you decide how many clusters to create when taking a “slice” like this? Again,
there's no fixed rule, but one method we like is to calculate the average number
of groups of cards created by the participants in the card-sorting study and then
try to approximate that.
After taking a “slice” through the tree diagram and identifying the groups cre-
ated by that, the next thing you might want to do is determine how those groups
compare to the original card-sorting data—in essence, to come up with a “good-
ness-of-fit” metric for your derived groups. One way of doing that is to compare
the pairings of cards in your derived groups with the pairings created by each
participant in the card-sorting study and to identify what percentage of the pairs
match. For example, for the data in Table 9.7 , only 7 of the 45 pairs do not match
those identified in Figure 9.4 . The 7 nonmatching pairings are apples-tomatoes,
apples-pears, oranges-tomatoes, oranges-pears, bananas-pears, peaches-toma-
toes, and peaches-pears. That means 38 pairings do match, or 84% (38/45).
Averaging these matching percentages across all the participants will give you a
measure of the goodness of fit for your derived groups relative to the original data.
MULTIDIMENSIONAL SCALING
Another way of analyzing and visualizing data from a card-sorting exercise is
using multidimensional scaling (MDS). Perhaps the best way to understand
MDS is through an analogy. Imagine that you had a table of the mileages between
all pairs of major U.S. cities but not a map of where those cities are located. An
MDS analysis could take that table of mileages and derive an approximation of
the map showing where those cities are relative to each other. In essence, MDS
tries to create a map in which the distances between all pairs of items match the
distances in the original distance matrix as closely as possible.
The input to an MDS analysis is the same as the input to hierarchical cluster anal-
ysis—a distance matrix, like the example shown in Table 9.8 . The result of an MDS