Special Topics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 233

o Savings gained from making changes earlier in the design
life cycle
o Reduced cost of providing training (if training is offered by
the company)

To illustrate some of the issues and techniques in calculating usability ROI,
we'll look at an example from Diamond Bullet Design (Withrow, Brinck, &
Speredelozzi, 2000). This case study involved the redesign of a state government
web portal. They conducted usability tests of the original website and a new ver-
sion that had been created using a user-centered design process. The same 10
tasks were used to test both versions. A few of them were as follows:

• Youareinterestedinrenewinga{state}driver'slicenseonline.

• Howdonursesgetlicensedin{thestate}?

• Toassistintraveling,youwanttofindamapof{state}highways.

• What4-yearcollegesarelocatedin{thestate}?

• Whatisthestatebirdof{thestate}?

Twenty residents of the state participated in the study, which was a between-
subjects design (with half using the original site and half using the new). Data col-
lected included task times, task completion rates, and various self-reported metrics.
They found that the task times were significantly shorter for the redesigned site, and
the task completion rates were significantly higher. Figure 9.11 shows task times for
the original and redesigned sites. Table 9.12 shows a summary of the task comple-
tion rates and task times for both versions of the site, as well as an overall measure
of efficiency for both (task completion rate per unit time).

Task Times (secs) for Old vs. New Site

300

250

200

150

Old Site
New Site

100

50

0

Figure 9.11 Task times for the original and the redesigned sites (an asterisk indicates significant
difference). Adapted from Withrow et al. (2000); used with permission.