Biomedical Engineering Reference
In-Depth Information
This chapter includes the presentation of (1) multimodal rendering techniques
for the interactive augmentation of otherwise neutral (i.e., flat, silent, and visually
homogeneous) ground surfaces; (2) multisensory effects and cross-modal illusions,
involving the senses of touch, kinesthesia, audition, and vision, that were made
possible by novel interfaces.
The chapter is organized as follows. Section
12.2
is dedicated to auditory render-
ing. Section
12.3
begins with the description of haptic rendering approaches, before
presenting multimodal systems and cross-modal approaches. Section
12.4
concludes
this chapter.
12.2 Auditory Rendering
12.2.1 Introduction
A walking task can be said to be intimately linked to a corresponding auditory task.
Not only do walkers constantly hear most of their own footsteps and foot movements,
but they are typically also aware of other persons walking in a shared auditory scene.
In parallel, the same scene may be populated by passive listeners who, while standing
or sitting, and not necessarily visually attending to pedestrians in their surroundings,
may nevertheless perceive the footsteps as part of the ambient soundscape.
These simple considerations already say much about the importance of the audi-
tory cues in informing ones perception and action loop during a walking task, and,
furthermore, in conveying information that can have social relevance when contribut-
ing to form a soundscape that is shared by several listeners.
As with any other type of non visual, ecological feedback, footstep sounds can
occupy the periphery of the attention. In other words, we need to make conscious use
of this feedback unless it brings to our ears salient cues, either familiar or unexpected.
A similar process happens for instance when a car driver's attention may be triggered
by an almost imperceptible change in the sound of the engine signaling potential
malfunctioning of the car, even after hours on a long trip along a monotonous highway
[
75
]. We do not need much quantitative science to establish these observations in
an empirical way: the use of footstep sounds as an auditory warning has long been
recognized by movie directors, who used to ask their Foley artist for preparing the
right walking sound when a new character entered the movie stage, or for sonifying
night-time chase actions that were typical of the “noir” genre.
It should be clear, at this point, that walking sounds are expressive. Through the
long familiarity with our own and others' footsteps, we built subjective mental maps
linking such sounds to corresponding physical attributes and gestures of the walking
person. Some of these links are obvious, and have been exploited for instance in early
computer game designs. All vintage electronic game players probably remember the
use of iconic footstep sounds to render the number and moving speed of the enemies
in Space Invaders™, a popular computer game of the late 1970s: the designers of
