Comparing more than two means: one factor ANOVA with a withinsubject design - Improving the User Experience through Practical Data Analytics - page 165

than Design 2, a scene of a young couple snuggling together under one umbrella during
a shower, with the Eiffel Tower bisecting a slate-gray sky. Furthermore, you were able
to report that “the low p -value of 0.023 means we have a statistically signiicant differ-
ence between the two designs. We should launch with Design 1.”
So far, so good at your new gig!! But, as soon as you drop a hard copy of the inal
report on Creative Director Kristen McCarthey's desk, she makes yet another request:
“It's been a long time since we've conducted any kind of testing on the general shop-
ping low, and in the meantime, we've made some changes. Can you get a test going asap?”
“What's your time frame,” you ask?
“Well, I'm meeting Massimo at 3pm Wednesday next week. Is there any way to
have results by then?”
You do some quick calculations in your head. It's Tuesday morning, a little over
a week until the meeting with CEO Massimo Ranieri.
“Well, if you don't mind me doing half-hour tests, I can do all the testing in one
day and just try to report out on the biggest problems.”
“Sure, sounds good. Don't make a federal case out of it. Just ind the low-hanging
fruit. ”
Because time is tight, you begin the recruit immediately, aiming for a Friday
test. Then, you start by performing a heuristic review using the persona of a typical
Mademoiselle La La client trying to make a dress purchase. After a morning
of studying the site, four potential problem areas pop out: (1) The ability to ind
the right size of dress, (2) The ability to add the dress to “My Bag” (inexplicably,
Mademoiselle uses “My Bag” instead of “shopping cart,”) (3) Entering credit card
information in the “Checkout” area, and (4) Choosing the best shipping method.
Based on those areas of interest, you start to create the tasks for the test. The tasks
seem pretty straightforward, but under the hood, you've carefully crafted each one to
either conirm or refute your hypothesis about a usability problem.
Once you've got a irst draft of the test script—which also includes your post-task
rating scales of ease-of-use—you send it to other members of the UX team as well as
the Creative Director. The recruitment is proceeding wonderfully because all you had
to do was send out an e-mail blast to current customers, offering them a $100 site-wide
credit for a 1-hour in-person usability test. Besides, the Mademoiselle La La brand is
pretty hot right now, and most of the hip urbane women are just curious about your
cool converted mill space in Boston's South End, and want to check out the place. Who
knows? They might be discovered as the next hot Mademoiselle La La model.
The long day of testing on Friday—10 participants, half-hour tests—proceeds
very well, albeit in a blur and without your getting a chance to eat lunch. No cancel-
lations, no no-shows, and articulate and thoughtful participants.
When you get to look at the data on Monday morning, your suspicions about
usability problems are conirmed, but there are some interesting twists.
Speciically, the completion rate for inding the right size dress is decent, as is
the completion rate for adding to “My Bag;” however, entering a credit card and