Information Technology Reference
In-Depth Information
Chapter 9
Experimental Setups for User
Evaluation of Mobile Devices
and Ubiquitous Systems
Francis Jambon
LIG / MultiCom, Grenoble University, France
ABSTRACT
Nowadays, mobile devices features are often linked up to the context of usage. As a consequence, re-
searchers must consider not only the user and the device, but also the surrounding environment when
designing effective user study evaluations. Two opposite experimental setups are possible: in-situ and
in the laboratory. There is no consensus on their respective benefits, for instance with regard to the
number of usability issues detected. In this chapter, the author isolates independent variables that could
contribute to evaluation biases by proposing a taxonomy that splits the in-situ and laboratory experi-
mental setups into two new setups. The author describes the concept of the “Uncertainty Principle” to
emphasize the dilemma between precise observation and bias minimization and introduce the “Trojan
Horse” technique to partially overcome the consequences of the uncertainty principle. As a conclusion,
a methodology using the four experimental setups in a complementary way is proposed.
INTRODUCTION
For instance, GPS-based automotive navigation
systems or network capable smartphones are
nowadays considered common objects. Moreover,
as mobile devices continue to evolve, researchers
have transitioned evaluating ubiquitous environ-
ments and pervasive systems from laboratory
prototypes to real-world implementations.
In the last decade, mobile devices have reached
the threshold of technical maturity to be widely
used in both professional and leisure contexts.
Search WWH ::
Custom Search