Touch and Speech: Multimodal Interaction for Elderly Persons - Biomedical Engineering Systems and Technologies - page 360

Touch and Speech: Multimodal Interaction
for Elderly Persons

Cui Jian 1 , Hui Shi 1 , Frank Schafmeister 2 , Carsten Rachuy 1 , Nadine Sasse 2 ,
Holger Schmidt 3 , Volker Hoemberg 4 , and Nicole von Steinbüchel 2

1 SFB/TR8 Spatial Cognition, Universität Bremen, Germany
{ken,shi,rachuy}@informatik.uni-bremen.de
2 Medical Psychology and Medical Sociology,
University Medical Center Göttingen, Germany
{frank-schafmeister,n.sasse,
nvsteinbuechel}@med.uni-goettingen.de
3 Neurology, University Medical Center Göttingen, Germany
[e-mail]
4 SRH Health Centre Bad Wimpfen, Germany
[e-mail]

Abstract. This paper reports our work on the development and evaluation of a
multimodal interactive guidance system for navigating elderly persons in
hospital environments. A list of design guidelines has been proposed and im-
plemented in our system, addressing the needs of designing a multimodal inter-
faces for elderly persons. Meanwhile, the central component of an interactive
system, the dialogue manager, has been developed according to a unified dialo-
gue modelling method, which combines the conventional recursive transition
network based generalized dialogue models and the classic agent-based dialo-
gue theory, and supported by a formal language based development toolkit. In
order to evaluate the minutely developed multimodal interactive system, the
touch and speech input modalities of the current system were evaluated by an
elaborated experimental study with altogether 31 elderly. The overall positive
results on the effectiveness, efficiency and user satisfaction of both modalities
confirm our proposed guidelines, approaches and frameworks on interactive
system development. Despite the slightly different results, there is no signifi-
cant evidence for one preferred modality. Thus, further study of their combina-
tion is considered necessary.

Keywords: Multimodal interaction, Elderly-centered system design,
Human-computer interaction, Spoken dialogue systems, Formal methods.

1 Introduction

Multimodal interfaces is gaining more and more importance for its promising possi-
bility to achieve a significantly more effective and efficient human computer interac-
tion (cf. [1]), they also increase users' satisfaction and provide a more natural and
intuitive way of interaction (cf. [2]). Meanwhile, due to the demographic development