Multi-modal Interaction in Biomedicine - Ambient Intelligence for Scientific Discovery - page 198

5 VR and Desktop: An Integrated Solution
The results of the experiments on user profiling led us to the idea to combine
VR and desktop projection modalities within the same interaction visualization
system. In this case different types of users will be able to work within the same
environment and switch between different projection modalities if necessary. We
see at least three possibilities of how this idea can be deployed.
One of the possibilities of how VR and desktop applications can be combined
is shown in Fig. 8. The main idea is to provide access to an already existing
desktop application by “absorbing” its GUI into VR [1, 4]. A desktop application
is represented in a separate window. For its activation and further manipulations
a user must use a wand and/or a keyboard, which is not very intuitive and quite
often leads to the significant meshing of the interaction process. Thus, if the
position of a 'desktop window' in VR is not fixed, it is very easy to loose it while
navigating in a 3D world.
Another possibility is to provide a user with an integrated workplace, where
he or she can work in the virtual environment and at a desktop PC at the
same time. Working alternating at a desktop PC and at a VR installation is the
typical situation for a programmer and also for a CAD-Designer (e.g., the PI-
casso system [19]). Because of repeatedly putting up and down input devices and
glasses and also due to repeatedly standing up and sitting down when changing
the workplace, this is very demanding and time consuming.
A Personal Space Station (PSS) is a relatively new concept for deploying the
interaction-visualization support [15]. The main advantage of a PSS is that it
initially combines the elements of VR and desktop projection modalities within
the same system and it is possible to switch between them if necessary.
A PSS allows users to interact directly with a virtual world. A PSS consists
of a semi-transparent mirror, in which a stereoscopic image is reflected. A user
reaches under a mirror to interact with the virtual objects directly with his or
her hands or by using task-specific input devices. Fig. 9 shows the experimental
setup of a PSS that has been built at the University of Amsterdam. By definition
a PSS is an individual environment, but there is a possibility to build a shared
environment where users can manipulate the same virtual objects working on
different PSSs [7]. More information about a PSS concept can be found in [24].
The idea to combine VR and desktop projection modalities on a PSS sounds
very attractive. However, its deployment is not an easy task. Now both the VR
and the desktop versions of the VRE system can run on a PSS. But to switch
a user has to restart a system, which is very uncomfortable and does not allow
using the functionality of both versions at the same time.
The interaction in VR and desktop projection modalities is different with
respect to navigation, locomotion, manipulation and measurement capabilities
[5]. To combine the Virtual Operating Theatre and a Personal Desktop Assistant,
a PSS has to support input and output modalities provided by both VR and
desktop simultaneously. This leads us to the development of a new concept of
'a multi-modal desktop-VR interaction'. This concept is based on the principle
of exploiting interaction capabilities of VR and desktop simultaneously without
changing devices and a workplace.