Case Studies - Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics - page 274

We then compared the number of
peaks per minute for each participant's
tasks using the digital textbook to tasks
using the printed textbook. Results showed
that the digital and the paper textbooks
had average peaks per minute of 6.2 and
7.6, respectively. However, using a paired
samples t test, this difference was not sta-
tistically significant ( p = 0.23) at a 95%
confidence interval (see Figure 10.26 ).
Figure 10.27 shows the average number of
peaks by participant for the two groups.
Comparing the peaks per minute across
the different tasks, the paper textbook had
higher peaks per minute than the iPad text-
book on six out of the seven tasks.

Average peaks per minute

10.0
9.0
8.0
7.0
6.0
5.0
4.0
3.0
2.0
1.0
0.0

Printed textbook Digital textbook

Figure 10.26 Average peaks per minute with 95% confidence limits for
printed textbook and digital textbook tasks.

Average peaks per minute

10.00
9.00

10.5.4 Qualitative Findings
Once we observed that average peaks
per minute were trending higher for
the paper textbook than the digital
textbook, we compiled the qualita-
tive feedback that we had collected
from the Microsoft Reaction Cards,
as well as the poststudy questions.
While participants described the
digital version as “organized,”
“easy to use,” and “efficient,” par-
ticipants described the paper text-
book as “slow,” “time-consuming”
and “old.” Figures 10.28 and 10.29
show word clouds for the paper text
book and digital textbook, respec-
tively. The larger the font, the most frequently the card was selected. The shade
of the text does not have any meaning.

8.00
7.00

6.00
5.00

4.00
3.00

2.00
1.00

0.00

A1 A2 A3 A4 A5 A6 A7 B1 B2 B3 B4 B5 B6 B7

Figure 10.27 Average peaks per minute per task broken out by group. Group A
used the digital textbook first; group B used the digital textbook first.

Affectiva Q Sensor data showed us that participants experienced higher
arousal while using the printed textbook, but combining those results with
qualitative data from the Microsoft Product Reaction Cards, as well as moder-
ated discussion, revealed that that the higher levels were due to negative emo-
tions from a difficulty in performing search and comprehension tasks.

At the end of the session, the moderators asked participants to choose
between digital and printed versions of the textbook; surprisingly, participants
were split with five preferring the digital and five preferring the printed text-
book. Although this was not the focus of our research, this split suggests that the