Appropriating Heuristic Evaluation for Mobile Computing - Human-Computer Interaction and Innovation in Handheld, Mobile and Wearable Technologies - page 29

Participants and Materials • Heuristic 6: Flexibility, efficiency of use

and personalization: Allow mobile users to
tailor/personalize frequent actions, as well
as to dynamically configure the system
according to contextual needs. Whenever
possible, the system should support and
suggest system-based customization if
such would be crucial or beneficial.
• Heuristic 7: Aesthetic, privacy and social
conventions: Take aesthetic and emotional
aspects of the mobile device and system
use into account. Make sure that users' data
is kept private and safe. Mobile interaction
with the system should be comfortable and
respectful of social conventions.
• Heuristic 8: Realistic error management:
Shield mobile users from errors. When an
error occurs, help users to recognize, to
diagnose, if possible to recover from the
error. Mobile computing error messages
should be plain and precise. Constructively
suggest a solution (which could also in-
clude hints, appropriate FAQs, etc). If
there is no solution to the error or if the
error would have negligible effect, enable
the user to gracefully cope with the error.

The study enlisted 8 usability experts 4 , as partici-
pants, to perform a HE of two mobile applications
for which we had already identified a number
of usability flaws. The two criteria we used to
select the applications to test were: being a typi-
cal application whose problems are known and
evident; application whose tasks are simple and/
or self-evident. After searching for applications
fulfilling the foregoing conditions, we chose the
following two applications: Appl.1) a mobile
device application in which location matters or
that primarily relies on mobility: we considered
a PDA-based supermarket application; Appl.2)
a mobile device application in which interface
navigation is key: we considered a web-based
freeware email application for PDAs 5 . We also
prepared the following materials for the evalua-
tors: consent form, demographics questionnaire,
post-evaluation form for participant's comments
(to be filled out by the study moderator), a set of
Nielsen's 10 usability heuristics, our proposed
set of mobile usability heuristics (Figure 2), and
Nielsen's five-point Severity Ranking Scale (SRS)
(Nielsen, 1994b) (which is described in Figure 3).

Assessing Heuristics Performance Experimental Conditions

To investigate the potential benefits of applying
our set of heuristics for the evaluation of mobile
applications, we devised and conducted an ex-
perimental study aimed at comparing the support
provided by our new set of mobile heuristics vs.
standard usability heuristics (here Nielsen's heu-
ristics) to experts performing heuristic evaluation
of mobile applications.

The experiment had the following two experi-
mental conditions:

• Condition 1: N. 4 experts individually
performed the HE by applying Nielsen's
standard set of heuristics and Nielsen's
SRS to both applications.
• Condition 2: N. 4 experts individually
performed the HE by applying our set of
mobile heuristics and Nielsen's SRS to
both applications.

Experimental Design

Here we describe various parameters pertaining
to the set-up or design of the experimental study.