Synthesis and Analysis Techniques for the Human Body: R&D Projects - 3D Modeling and Animation: Synthesis and Analysis Techniques for the Human Body - page 369

The introduction of virtual reality will allow experimenting with new pedagogical
mechanisms for teaching music and, more specifically, for correcting hand
positioning. Entirely different from the adoption of simple videotapes, by using
the VR it will be possible for the pupil to re-execute specific passages that are
not planned in the video, but can be directly produced on the basis of the music
score or MIDI. This is a high-level coding for the finger movements. The high-
level coding allows the updating of the pupil database via Internet. It would be
impossible to send MPEG videos to show the same gestures.

IMUTUS technical approach

Many researchers have worked on 3D finger representation, but their concen-
tration was mainly on gesture recognition aspects, providing no timing or
synchronization facilities for the movement of the fingers. The major problem in
3D finger representation and playback synchronization is the correct timing of
the virtual fingers.
Concerning the gesture description, using separate controls for each geometry
can be useful, but this increases the complexity of the system. The controls can
be reduced using inverse kinematics methods by implementing only one control
for each finger. For the gesture description subcomponent, a hybrid system with
inverse kinematics and H-Anim models is adopted. The 3D finger representation
module supports the following functions:

• Read initial position values (read position values for fingers).

• Read final position values (read position values for fingers).

• Get morphing total time (read time for morphing from initial to final state).

• Calculate intermediate position values of the morphing sequence (real-time

finger position calculation and interaction with the 3D rendering engine).

Correct representation and modeling of the virtual hands can also be performed
using virtual reality gloves (e.g., the CyberGlove of Immersion Technologies)
(Immersion Corp., 2002). CyberGlove is a low-profile, lightweight glove with
flexible sensors which can measure the position and movement of the fingers and
wrist. It is based on a resistive, bend-sensing technology and its sensors are thin
and flexible enough to produce almost undetectable resistance to bending. The
device can provide accurate measurements for a wide range of hand sizes.