Self-Reported Metrics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 155

their trust of websites based on very brief exposures to images of web pages. They
used 50 screenshots of popular financial and health-care websites. After viewing
a page for only 50 msec, participants were asked to give a rating of their trust of
the site on a 1 to 9 scale. After a break, they repeated the procedure in a second
trial with the same 50 images. They found a significant correlation ( r = 0.81,
p < 0.001) between the trust ratings in the two trials.
Several years ago, Tullis conducted an online study of 10 different websites to
learn more about what makes a website engaging . He defined an engaging web-
site as one that (1) stimulates your interest and curiosity, (2) makes you want to
explore the site further, and (3) makes you want to revisit the site. After explor-
ing each site, participants responded to a single rating worded as “This website
is: Not At All Engaging … Highly Engaging” using a five-point scale. The two
sites that received the highest ratings on this scale are shown in Figure 6.22 .
Figure 6.22 Websites rated as the most engaging of 10 sites studied.
One of the techniques often used in analyzing data from subjective rat-
ing scales is to focus on the responses that fall onto the extremes of the scale:
the top one or two or bottom one or two values. As mentioned earlier, these
are often referred to as “Top Box” or “Bottom Box” scores. We used this tech-
nique in an online study assessing users' reactions to various load times for an
intranet homepage. We manipulated the load time artificially over a range of
1 to 11 seconds. Different load times were presented in a random order, and
users were never told what the load time was. After experiencing each load time,
users were asked to rate that load time on a five-point scale of “Completely
Unacceptable” to “Completely Acceptable.” In analyzing the data, we focused
on the “Unacceptable” ratings (1 or 2) and the “Acceptable” ratings (4 or 5).
These are plotted in Figure 6.23 as a function of the load time. Looking at the
data this way makes it clear that a “crossover” from acceptable to unacceptable
happened between 3 and 5 seconds.