Information Technology Reference
In-Depth Information
command set comprises 18 commands that our
user-based studies have shown to be relatively easy
to recall. Results have also shown that the kinds
of functions iGraph-Lite offers are relevant to
people using graphs in their daily work. Adopting
a user-centred design methodology (e.g. Preece et
al., 2007), iGraph-Lite has thus been tested and
revised iteratively in terms of usefulness and us-
ability on both blind and sighted participants. It
has been shown to facilitate the construction of a
correct and accurate mental model of single-line
graphs for both types of audiences (Ferres et al.,
2010) provided the information is presented in a
sequence that allows the person to visualize the
structure and data labels before dealing with the
data in the graph. iGraph-Lite avoids the prob-
lem of recognizing different accents correctly
that speech-based technologies typically face.
However, natural language interactions using an
unrestricted vocabulary are still some distance
away from implementations available to the mass
market.
provided many examples showing that controlled
studies are most usefully applied in the very early
stages of the evolution of a new technology.
Since Egon Brunswik (1956) proposed that
we need to consider the environment carefully in
which people work, play, and make decisions as
well as studying their tasks and task performance,
this has been taken seriously in the Human Fac-
tors literature. Along those lines, job shadow-
ing, contextual interviews (Beyer & Holtzblatt,
1998), and other forms of 'direct' observation
techniques (Drury, 1995) in the users' context
are warmly advocated in the usability literature
(e.g. Preece et al., 2007). However, in the world
of mobile devices, many of which are used only
occasionally, often in situations in which it would
be impossible or too intrusive for researchers to
be present, these techniques do not work (see e.g.
Bennett et al., 2006; Svanæs et al., 2008). Even
if target users are readily accessible, it may be
impossible to follow them around and observe
them unobtrusively in their 'natural environment'
to collect the kinds of 'ecologically valid' data that
Brunswik so eloquently advocated. In addition,
the recording of observations presents an obvious
challenge. Video recording of mobile device usage
capturing button presses and screen shots are dif-
ficult to employ without disturbing normal usage
patterns, and video data can be difficult to analyze
post hoc because there are likely to be gaps in the
recording due to screen blockages and the like.
Potential solutions to the problem of collect-
ing observation-based data are reported in the
literature, but many of these suffer from obvi-
ous limitations. For example, it is difficult to
imagine using a wireless camera that clips onto
a mobile device being used in outdoor situations.
In one reasonably widespread approach, the user
under observation wears a backpack containing
the necessary recording equipment (Roto et al.,
2004). While this is less intrusive than the wireless
clip-on camera, it is still bothersome and quite
heavy. The solution suggested by Reichl and his
colleagues (2007) in which the user wears a hat
SOME THOUGHTS ON DATA
COLLECTION IN MOBILE
USABILITY EVALUATIONS
According to Cockton (2008): “usability evalu-
ation measures remain too close to what were
originally independent variables in factorial exper-
iments. The basis for genuine usability problems
in such variables is not guaranteed, but there has
been little progress in finding replacements since
HCI's shift from the laboratory to field studies” (p.
287). This widespread view coincides with that of
Benyon and his colleagues (2010). We do agree
that novel data collection and usability evalua-
tion methods are needed in the mobile arena, but
we argue that it is not an 'either-or' (controlled
quantitative laboratory studies versus qualitative
field studies) situation. In our view, both field-
and laboratory studies have a legitimate place in
mobile HCI. Indeed, the above review has clearly
Search WWH ::
Custom Search