Special Topics - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 236

• Auser-centeredredesignof Staples.com resulted in 67% more repeat customers

and a 10% improvement in ratings of ease of placing orders, overall purchasing
experience,andlikelihoodofpurchasingagain.Onlinerevenueswentfrom$94
millionin1999to$512millionafterimplementationofthenewsite(Human
Factors International, 2002).

• Amajorcomputercompanyspent$20,700onusabilityworktoimprovethe

sign-on procedure in a system used by several thousand employees. The resulting
productivityimprovementsavedthecompany$41,700thefirstdaythesystemwas
used (Bias & Mayhew, 1994).

• Afteraredesignofthenavigationalstructureof Dell.com , revenue from online

purchaseswentfrom$1millionperdayinSeptember1998to$34millionperday
in March 2000 (Human Factors International, 2002).

• Auser-centeredredesignofasoftwareproductincreasedrevenuebymorethan80%

over the initial release of the product (built without usability work). The revenues
of the new system were 60% higher than projected, and many customers cited
usability as a key factor in deciding to buy the new system (Wixon & Jones, 1992).

9.5 SUMMARY
Here are some of the key takeaways from this chapter.

1. If you're dealing with a live website, you should be studying what your

users are doing on the site as much as you can. Don't just look at page
views. Look at click-through rates and drop-off rates. Whenever possi-
ble, conduct live A/B tests to compare alternative designs (typically with
small differences). Use appropriate statistics (e.g., χ 2 ) to make sure any
differences you're seeing are statistically significant.

2. Card sorting can be immensely helpful in learning how to organize some

information or an entire website. Consider starting with an open sort and
then following up with one or more closed sorts. Hierarchical cluster anal-
ysis and multidimensional scaling are useful techniques for summarizing
and presenting the results. Closed card sorts can be used to compare how
well different information architectures work for the users. Tree-testing
tools can also be a useful way to test a candidate organization.

3. Accessibility is just usability for a particular group of users. Whenever

possible, try to include older users and users with various kinds of dis-
abilities in your usability tests. In addition, you should evaluate your
product against published accessibility guidelines or standards, such as
WCAG or Section 508.

4. Calculating ROI data for usability work is sometimes challenging, but

it usually can be done. If the users are employees of your company, it's
generally easy to convert usability metrics such as reductions in task
times into dollar savings. If the users are external customers, you gener-
ally have to extrapolate usability metrics such as improved task comple-
tion rates or improved overall satisfaction to decreases in support calls,
increases in sales, or increases in customer loyalty.