Database Reference
In-Depth Information
CHAPTER
5
Pass or fail? Binomial-
related hypothesis testing
and conidence intervals
using paired samples
5.1
INTRODUCTION
We noted in Chapter 4 and repeat here that task success is a fundamental metric for any
UX researcher who regularly conducts usability testing. Thus, we indicated that pass/fail
is a fundamental metric that you should always deliver for each task when reporting your
results. In this chapter, we discuss pass/fail data for a within-subject design, or paired data.
5.2
CASE STUDY: CAN I REGISTER FOR A COURSE
AT BACKBOARD.COM?
You've just landed a new gig as a UX researcher at Backboard, a learning management
system (LMS) company that provides software for the administration, documentation,
tracking, reporting, and delivery of online education courses and training programs.
Backboard is barely out of start-up phase, but you're still surprised to ind out that
no formal usability testing was ever conducted during development of their lagship
product Backboard LMS.
During your irst week on the job, you run into Bob Buzzkill, the chief technology ofi-
cer, at the coffee machine. Buzzkill has worked in hi-tech for a long time; he brandishes
a pocket protector and digital watch to prove it. Since formal usability testing was rarely
if ever conducted during his formative years in the 70's and 80's, he's sceptical of the
entire process, but has begrudgingly begun to accept its value. Since you know he'd never
introduce himself to a practitioner of a discipline he barely acknowledges, you decide to
take the initiative. You introduce yourself, and he sheepishly admits that “our usability was
kinda ad hoc,” but they shipped the software anyway because they had contracts to fulill.
“Can we obtain some real usability data asap?” Buzzkill asks. “We're hearing
some rumblings about usability problems with our largest customer…. I want them
to know we're at least working on it!”
You're off to the races. First, you perform a heuristic review using the persona of a typ-
ical company employee-learner who is perusing training offerings, registering for courses,
and getting on waiting lists. Based on the review, you have hypotheses of what usability
issues exist in the product. Based on those hypotheses you start to create the tasks for a
usability test. The tasks seem pretty straightforward, but under the hood you've carefully
crafted each one to either conirm or refute your hypothesis about a usability problem.
Search WWH ::
Custom Search