Information Technology Reference
In-Depth Information
2 Multi-modal Interaction and Projection Modalities
Traditionally, multi-modal interaction is considered as interacting with a com-
puter system using more than one input or output modality at a time, usually
suggesting drastically different modalities to be used simultaneously [16]. The
simplest example of multi-modal interaction is the simultaneous use of a mouse
and a keyboard. More advanced multi-modal interfaces may combine voice input
with a mouse and/or tactile feedback.
Today's advanced computer technologies provide different forms of input/out-
put modalities. The possibility to combine them while using the same applica-
tion leads to the development of a multi-modal interaction style. We may use
command dialogue, speech recognition, data entry, graphics, web and pen-based
interfaces, direct manipulation, haptics, gestures and even interacting via GPRS
enabled cell phones. For example, one can draw simple images by walking round
the streets of a city and entering data points along the way via a cell phone [25].
Virtual and augmented reality can be also considered as input/output modalities
used for providing the interaction-visualization support [16]. These two relatively
new interaction paradigms are alternatives to a desktop solution applied on a
common PC (or a PDA). They are usually referred to as projection modalities [6].
Virtual reality (VR) is a projection modality, invented in 1965 by Ivan Suther-
land [20] and intended to make the interaction process with a computer more
intuitive and appealing. The main difference of an immersive 1 [5] application
in comparison to a desktop one is that it can provide the user with a sense of
presence. In VR an artificial world is created around the user, which gives the
impression of being in that world and able to navigate through and manipulate
objects in the world [5]. Ideally, VR has to provide an environment, where users
can interact freely in a 3D space. However, in practice the utilization depends on
hardware and software solutions chosen for deploying this projection modality.
When VR is combined with the real world, this projection modality is called
augmented reality (AR). AR is a combination of a real scene viewed by a user and
virtual objects generated by a computer that augment the scene with additional
information. So if in VR the artificial world is generated completely, in AR the
real world is combined with elements from the artificial one [11]. Actually AR is a
projection modality, which is the closest to the real world because a user mainly
perceives the real world with just a bit of computer-generated data (Fig. 1). As
for a desktop, we here refer to a conventional PC. A conventional PC is highly
refined to support oce work, which is characterized by a user sitting in a chair,
at a desktop preferably with a lot of space for a keyboard and a mouse [15].
The user is typically situated at the same desktop the entire day and primarily
works alone. In the case of a desktop projection modality, a 3D environment is
projected on a computer screen (Fig. 2) and users' manipulation and navigation
capabilities become limited within a 2D projected world.
1 Immersive VR offers the user a stereoscopic, head tracked, as much as possible
surrounding visual experience using either head-mounted displays or (multiple) pro-
jection screens, such as in the CAVE environment. Such systems are deemed 'semi-
immersive', when an all around picture is not offered [5].
Search WWH ::




Custom Search