Case Studies - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 257

201 2 ) after every task and the System Usability Scale at the end of the test. The
SEQ is a single question asking participants to rate the difficulty of the task on a
seven-point scale, where 1 is “very difficult” and 7 is “very easy.”
We collected the data using Excel spreadsheets based on a template we use
for every usability test that includes formulas for calculating our metrics. As a
result, we can report the quantitative results almost immediately after finishing
a study. Since we had product release goals, the team was very interested in hear-
ing quickly if we were getting closer to our goals.
The only downside with our spreadsheet methodology is that one of our
experimenters had a difficult time with the spreadsheet's keyboard shortcuts in
the beginning of these studies. As a result, we lost many time-on-task measure-
ments and do not have enough data to present those findings here. However,
we often see interesting differences in time on task, even when testing mockups.
10.3.3 Workflow
For the purposes of this case study, we focused on one aspect of the project:
workflow design and results. The workflow task in all four rounds of usability
testing was essentially the same: accept a workflow task assigned to the partici-
pant, approve the page being edited, add a note to the workflow, and finish the
workflow task assigned. Task instructions given to the participant were:
You've received an automatic workflow item! When you made changes to the
homepage, a workflow automatically triggered and sent a workflow to you to
approve the page. Please complete the workflow and add the note “fixed typo,
changed image” so that your editor knows what you've changed.
Workflow was a difficult task for us to get right from a design perspective. In
the new slim design, we had very little room to work with to communicate com-
plex task requirements, and we had to work within the existing system limita-
tions. There wasn't time or resources to allow for a complete rewrite of the code.
Figures 10.11-10.15 show screenshots of the design and how it changed over the
four testing iterations. We started out with a very modular approach, requiring
the user to find each piece of the functionality on the toolbar and ended with
a more “wizard”-like approach where the user is guided through the process.
In each version of the design, the toolbar is shown at the bottom of the screen
in gray. The “task panel” (or later “Task Inbox”) is the blue panel just above
the toolbar on the left-center portion of the screen. Early versions of the design
(1 and 2) included some task functions (“Accept Task”) within the task panel,
while later versions (3 and 4) moved those actions into a second “Task Editor”
window (the panel on the right side of Design 3, and Screen 2 for Design 4). The
designs, along with a brief description, appear in Figures 10.11-10.15 .
WORKFLOW DESIGN 1
The first design was focused on trying to stay with the very modular design of
the new toolbar. The toolbar can be seen at the bottom of Figure 10.11 with the