Information Technology Reference
In-Depth Information
8
Integrating Spatial Audio, Visual, and
Gestural Interfaces for Supporting
Situated Co
llaborative Work
Takeshi Nakaie, Takayuki Koyama, and Masahito Hirakawa
CoNTENTS
8.1 Introduction ................................................................................................ 180
8.2 Related Work .............................................................................................. 181
8.3 System Overview ....................................................................................... 182
8.3.1 Organization of the System .......................................................... 182
8.3.2 Processing Flow ............................................................................. 184
8.3.3 Primitive Gestures ......................................................................... 184
8.4 Implementation .......................................................................................... 186
8.4.1 Capturing of Gestures................................................................... 186
8.4.2 Positioning of Sounds.................................................................... 187
8.4.2.1 Algorithm......................................................................... 187
8.4.2.2 Evaluation......................................................................... 188
8.5 Application to Computer Music Creation .............................................. 191
8.6 Conclusion .................................................................................................. 193
References............................................................................................................. 193
Ubiquitous computing, or pervasive computing, is an increasingly popular
technique for developing new human-computer interfaces and computer
applications. Computers watch what happens in a physical space where
people are and serve them with meaningful information. Here, most of the
systems rely on a visual media for user navigation, while there are some
other interesting interaction media. In this paper we present a collabora-
tive multimodal system toward spatial presence and perceptual realism of
sounds. A two-dimensional sound display called Sound Table is its central
component, in which 16 speakers are mounted so that all of the users have
the sound experience in common. A stick-shaped device is provided as a
means for interaction with the system. The system recognizes manipulation
of the device in a 3-D space over Sound Table and then gives users feedback
through both visual and auditory channels.
179
