Special Topics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 213

independent samples, the result ( p = 0.41) would not have been anywhere near
significant. (See Chapter 2 for details on how to run a paired t test in Excel.)

9.1.2 Click-Through Rates
Click-through rates can be used to measure the effectiveness of different ways
of presenting a link or button. They indicate the percentage of visitors who are
shown a particular link or button who then actually click on it. If a link is shown
100 times and it is clicked on 1 of those times, its click-through rate is 1%. Most
commonly the term is used to measure the effectiveness of web ads, but the
concept applies to any link, button, or clickable image. For example, Holland
(2012a) describes a study of two different buttons on a product page for an
ecommerce site, as shown in Figure 9.2 . The only difference between the two
pages was the wording of the green button: “Personalize Now” vs “Customize
It”. The click-through rate for the button labeled “Personalize Now” was 24%
higher. They continued tracking through to actual sales and found that clicks on
that version of the button also resulted in a 48% higher revenue per visitor. Why
did the “Personalize Now” text yield more clicks and more sales? We could spec-
ulate, but we don't really know. That's one of the limitations of live-site data.

Figure 9.2 Example of two button designs tested on a product page.

What analyses can be used to determine if the click-through rate for one link
is significantly different from that for another link? One appropriate analysis is
the χ 2 test. A χ 2 test lets you determine whether an observed set of frequencies
is significantly different from an expected set of frequencies. (See Chapter 2 for
more details.) For example, consider the data shown in Table 9.2 that represent

Table 9.2 Click rates for two different links: The number of times each link
was clicked and the number of t i mes each was presented but not clicked.
Click

No Click

Link #1 145 10289

Link #2 198 11170

Table 9.2 Click rates for two different links: The number of times each link was clicked and the number
of times each was presented but not clicked.