Information Technology Reference
In-Depth Information
used because they distract the user for his/her
task. Questionnaires can be used at the end of
the experiments, but they are not well adapted
to the evaluation of precise usability problems:
users forget them. In practice, the observation of
the interactions between the user and the device
can only be performed via computer logs. In
summary, experiments can be set up in-situ (1)
without ecological context but with plenty of data
available or (2) with ecological context but not
with enough data to interpret the user interaction
with the device.
We name this constraint the “Uncertainty
Principle” because of its semantic proximity
with the principle stated by Heisenberg con-
cerning the quantum physics. In our context,
this principle can be stated as follows: “it is not
possible, at the same time, to precisely observe a
situation of man-machine interaction without, by
side-effects, disturbing it.” Although it is known
that the observer affects the observed (Schulte-
Mecklenbeck & Huber, 2003), in our knowledge,
a similar principle has never been formalized for
mobile usability testing. The experimenters have
a difficult dilemma: either they choose to observe
with precision and accept biases, or they choose
to minimize biases, and in consequence accept
to have limited observations that minimize the
interest of the experiment.
Trojan Horse technique. Even if it is not a priori
possible to transgress the Uncertainly Principle,
we defined a technique that minimizes the biases
introduced by observation while guaranteeing
enough information collection to analyze the
interactions. Users cannot ignore that they are
being observed, but researchers can introduce a
recording device that has both visible and hidden
recording features. We first used this easy way in
the E-skiing experience. For this experiment, it
was not possible to use a shoulder camera hold
by a cameraman or a gooseneck camera above
the smartphone because of the ecological context.
So, we fastened a mini-camera on the skiers' hel-
mets. The video replay feature of the smartphones
justified the presence of the camera. Actually, the
main feature of the camera was to record the user
interaction with the smartphone, witch is supposed
to be in the camera's field of view. This easy way
was generalized in the Museum experiment. In
that experiment, we used a postcard as a guide for
the exhibition, but the postcard was also a RFID
based tracking system.
The Trojan Horse technique, as we named it,
takes as a starting point the Wizard of Oz idea of
masking to the user some aspects of the experimen-
tal setup. The Trojan Horse technique is based on
the double usage of a recording device. The first
usage is clearly visible and has a believable utility.
The second usage is masked and aims at recording
useful information for the observation. The basic
principle is to make the user accept a recording
device for a personal usage -the horse- in the
ecological context -the town of Trojan- in order
to put silently in that context a recording device
useful for observation -the Greek soldiers. In
practice, the Trojan Horse technique may be used
in many situations. Two approaches are possible:
•
If the mobile device or the ubiquitous en-
vironment already has a device that could
capture the needed information, it is pos-
sible to add new features to the device, for
instance a recording system.
•
If the mobile device or the ubiquitous en-
vironment does not have such a device, it
must be added. To do so, a believable us-
age must be found to it, in order to justify
its presence. This first usage does not need
to be useful for the experiment, it must just
be believable.
For instance, to study the geo-localized use
of SMS on mobile phones, the phones could be
equipped with a GPS. The GPS may be presented
to the test users as one of the tested features of
the phones, but in fact, it is used to detect the
user position when sending SMS. Nevertheless,
the Trojan Horse technique could not always
Search WWH ::
Custom Search