Information Technology Reference
In-Depth Information
When compared to other expert techniques,
such as guideline-based methods and cognitive
walkthrough, heuristic evaluation is strong in
terms of thoroughness (percentage of problems
found), but weak in terms of efficiency (number
of true positives vs. false positives) and, like other
inspection methods, is vulnerable to expert biases
(Cockton et al., 2003).
with elements of evaluation. The analysis
entailed documenting, for each of the pa-
pers, appropriate values for the following
dimensions:
Evaluation goal: is the evalua-
tion mainly intended to demonstrate
whether one technique is better than
another, that is, a comparative study;
or is it mainly exploratory, that is,
understanding what kind of usabil-
ity problems may rise with a given
design.
Evaluation method: is the evaluation
method expert-based (made by ex-
perts through inspection), user-based
(observing users performing tasks),
or model-based (computing usability
metrics through formal models).
Evaluation setting: is the evaluation
conducted in a laboratory (or any oth-
er controlled setting) or in the field.
Real device/emulator: is the ap-
plication under inspection tested
with a real device or in a emulated
environment.
Location matters: does the appli-
cation take location into account or
not. Moreover, each of the usability
researchers individually documented
mobile usability issues that were in-
dicated by (or evident from) each of
the papers. At the end of this process,
three different lists of usability issues
were produced, containing the analy-
sis of all the papers collected in the
first phase.
2. In the next step, the usability researchers
came together and consolidated their indi-
vidual realizations. Individual findings had
to be cross-checked and merged into a single
consolidated list. This was done in the form
of a spreadsheet.
3. Each researcher was then given the same
realized list of mobile usability issues and
APPROPRIATING USABILITY
HEURISTICS
Analyses of heuristic evaluation (HE) have shown
that it is more likely for this method to miss relevant
usability problems when the system to be evaluated
is highly domain-dependent, and when evaluators
have little domain expertise 2 . To overcome these
limitations of the method when applied to mobile
systems and settings, we have conducted an in-
depth investigation of usability issues affecting
mobile applications. The work leading to the set
of specialized heuristics for mobile computing
presented in the section entitled Methodology for
Realizing Mobile Heuristics is based on this em-
pirical evidence. The goal of the mobile heuristics
described in that section is to better support and
contribute to the domain expertise of evaluators
applying HE to mobile computing.
Methodology for Realizing
Mobile Issues
To develop usability heuristics for mobile com-
puting, three authors of this article worked as
usability researchers at the following activities:
1. Each of the three was assigned a unique set of
papers to analyze independently. The papers
originated from the list used in Kjeldskov
et al. (2003); a recent meta-analysis of HCI
research methods in mobile HCI 3 . We up-
dated the list with papers published in the
period 2004-2005 and selected only those
Search WWH ::




Custom Search