Information Technology Reference
In-Depth Information
Traditional music notation as an abstract representation of a performance it is
not sufficient to store all the information and data required to reconstruct the per-
formance with all the specific details. In order to keep an IMP performance alive
through time, not only its output, but also the whole production process to create the
output needs to be preserved.
22.5.2 Interactive Multimedia Performance (IMP) Systems
In this section we describe several different IMP systems and software with different
types of interaction and different types of data while the following section explains
how the CASPAR framework is used for their preservation.
22.5.3 The 3D Augmented Mirror (AMIR) System
The 3D Augmented Mirror (AMIR) [
230
,
231
] is an example IMP system which
has been developed in the context of the i-Maestro project (
www.i-maestro.org
)
[
232
], for the analysis of gesture and posture in string practice training. Similar to
many other performing arts, string players (e.g. violinist, cellists) often use mirrors
to observe themselves practicing to understand and improve awareness of their play-
ing gesture and posture. More recently, video has also been used. However, this is
generally not effective due to the inherent limitations of 2D perspective views of the
media.
The i-Maestro 3D Augmented Mirror is designed to support the teaching and
learning of bowing technique, by providing multimodal feedback based on real-time
analysis of 3D motion capture data. Figures
22.9
and
22.10
show screenshots of the
i-Maestro 3D Augmented Mirror interface which explore visualization and sonifica-
tion (e.g. 3D bow motion pathway trajectories and patterns) to provide gesture and
posture support. It uses many different types of data including 3D motion data (from
a 12-camera motion capture system), pressure sensor, audio, video and balance.
The i-Maestro AMIR multimodal recording, which includes 3D motion data,
audio, video and other optional sensor data (e.g. balance, etc) can be very useful
to provide in-depth information beyond the classical audio visual recording many
different purposes including technology-enhanced learning, and in this context for
the preservation of playing gesture and style for detailed musicological analysis
(now and in the future).
22.5.4 ICSRiM Conducting Interface
The ICSRiM Conducting System is another IMP system example. It has been devel-
oped for the tracking and analysis of a conductor's hand movements [
233
,
234
]. The
system is aiming at supporting students learning and practicing conducting, and
