Multi-modal Interaction in Biomedicine - Ambient Intelligence for Scientific Discovery - page 189

3.1 The Virtual Operating Theatre

We called the interaction style of the VRE system deployed for the VR projection
modality 'the Virtual Operating Theatre', because a user 'plays a role' of a
physician applying the treatment of a vascular disease on a simulated patient [2].
To support better the user's interaction within an operating theater, a multi-
modal interface [18] to the VRE system has been built. It combines context
sensitive interaction by voice and manipulation of 3D virtual objects using a
wand 4 and hand gestures. Although, the main VRE functionality can be accessed
both via a direct selection or manipulation and via a voice command, for time
consuming procedures related to grid editing, interactive measurements and data
exploration the direct manipulation technique remains the most reliable.
For the end-user of the VRE system, grid editing is the most important func-
tionality, since it permits to simulate the surgical procedure of the placement of a
bypass 5 on an artery. In VR users are capable to manipulate 3D objects directly.
They deal with 3D representations of an artery and a bypass. For representing
a bypass we use spline primitives. So the procedure of adding a bypass comes
down to re-scaling of a spline and positioning it correctly on an artery. These
manipulations are conducted using a wand. The same procedure can be applied
to the placement of a stent within an artery in the case of aneurism.
Measurements are crucial both for diagnosis and for planning the treatment.
Clinical decision making relies on evaluation of the vessels in terms of a degree
of narrowing for stenosis and dilatation 6 for aneurysms. The shape, length, di-
ameter, and even material of a bypass or a stent depend to a great extent on
the size and geometry of an affected vessel.
Interactive measurements in VR are organized as follows. For conducting a
measurement, a user has to position an appropriate number of active markers
on an object. Markers are building blocks of the distance, angle and line-strip
measurements. The number of necessary active markers depends on a measure-
ment to be done. For measuring a distance, a user has to add two markers and
if it is an angle, three. For conducting line-strip or tracing measurements - at
least two [22]. In VR a user can add a marker via direct manipulation using the
position and orientation of a wand.
A free clipping engine [12], which has been developed recently as a part of
the VRE system, is an interactive component aimed to help in the exploration
of big data sets. By restricting a view via a clipping plane the user may look
inside the patient's body or specific part of an artery. In VR a user may change
the orientation of a clipping plane by changing the direction of a wand and as a
result see the original data, obtained from a scanner, slice by slice (Fig. 4).

4 A wand is a hand driven controller combined with tracking sensors to allow the VR
system to receive user commands and to track the position of the hand with respect
to virtual objects; provides 6 degrees of freedom (position and orientation) [22].
5 A bypass is a graft rerouting a blood flow around blockages. Usually it is a piece of
vein taken from elsewhere in the body or an implant made from an organic mate-
rial [21].
6 Dilatation is an increase over the normal arterial diameter [21].