Information Technology Reference
In-Depth Information
Chapter 8
Large Scale User Trials:
Research Challenges and
Adaptive Evaluation
Scott Sherwood
University of Glasgow, UK
Stuart Reeves
University of Nottingham, UK
Julie Maitland
National Research Council of Canada, Canada
Alistair Morrison
University of Glasgow, UK
Matthew Chalmers
University of Glasgow,UK
ABSTRACT
The authors present a reflection on a series of studies of ubiquitous computing systems in which the
process of evaluation evolved over time to account for the increasing difficulties inherent in assessing
systems 'in the wild'. Ubiquitous systems are typically designed to be embedded in users' everyday lives;
however, without knowing the ways in which people will appropriate the systems for use, it is often in-
feasible to identify a predetermined set of evaluation criteria that will capture the process of integration
and appropriation. Based on the authors' experiences, which became successively more distributed in
time and space, they suggest that evaluation should become adaptive in order to more effectively study
the emergent uses of ubiquitous computing systems over time.
INTRODUCTION
moving from the relative safety of the usability
lab into the uncontrolled environment of everyday
life. For example, unpredicted contexts of use
and environmental features such as intermittent
network connectivity may challenge traditional
evaluation methods, and yet we gain the mobility,
When working with ubiquitous computing (Ubi-
comp) systems, challenges and rewards arise from
Search WWH ::
Custom Search