Pen-Based Interaction for Intuitive Music Composition and Editing - Intelligent Music Information Systems: Tools and Methodologies - page 265

state of the art In pen-Based
musIcal score composItIon
and edItIng

interpret subsets of these symbols, for instance
one recognizer to discriminate between clefs, one
to discriminate between accidentals, and so forth.
As a consequence, the first difficulty consists in
interpreting the user hand-drawn strokes, which
is furthermore complicated by the fact that every
scripter has its own way to realize a musical score
and draw the musical symbols. Finally, the fact
that some of these symbols have the same shape
(for instance a whole note, a half note, the figure
“0”, the character “o”) complicates even more the
problem. It is then essential to take into account the
context in which a stroke has been drawn in order
to interpret it: depending on the context, which
can be for instance structural and/or temporal,
the same drawing is interpreted differently (for
instance, a circle on a staff is more likely a head
note, whereas a circle below a staff is more likely
part of lyrics, i.e., the character “o”).

Before going further, it is interesting to notice
that, so far, there are little pen-based software
for musical notation composition and editing. In
particular, as far as we know, only systems for
classical musical score notations exist.
In this section, we first present the main
difficulties that make pen-based musical score
editors complex software to develop. We would
like to note that these problems also exist in other
pen-based structured document composition and
editing software. Then, we present the existing
approaches and highlight the way they deal with
each of these problems.

Difficulties in Developing Pen-Based
musical score editors

Dealing with Multi-Stroke Symbols

The two main difficulties in the development of
pen-based musical score editors are the interpreta-
tion of the user strokes in the context of structured
documents and the management of symbols drawn
with more than one stroke.

Some of the classical musical symbols cannot
be, or are not traditionally, drawn with only one
stroke: they are called multistroke symbols (in
opposition to unistroke symbols). For instance,
the sharp symbol is constituted of two horizontal
segments and two vertical segments; as a conse-
quence, such a symbol is classically drawn with
four strokes. This presents a second difficulty
for the eager interpretation process. Indeed, the
system is faced with the dynamic segmentation
problem: it has to eagerly decide if a stroke is suf-
ficient to form a symbol, and interpret it as such,
or if it should wait for the following strokes. Once
again, the solution can be based on the exploitation
of structural and/or temporal contexts.
Some authors avoid part of or all these prob-
lems, for instance by assuming that each musical
symbol has already been isolated before trying
to interpret it. Thus, George (2003) proposed to
exploit artificial neural networks in order to rec-
ognize already segmented symbols. But the use of
this method in an online musical notation editor

Interpreting the Strokes of Structured
Documents

Musical scores contain a lot of symbols of various
natures, such as clefs (G-clefs, C-clefs, F-clefs,
etc.), notes (whole-notes, half-notes, quarter-notes,
etc.), accidentals (flat, sharp, natural, etc.), figures
(time signature, etc.), characters (dynamics, lyr-
ics, etc.). Powerful hand-drawn shape recogniz-
ers can be developed to interpret these symbols
(Plamondon & Srihari, 2000), but we have to
take into account the fact that the more symbols
a recognizer has to discriminate, the less robust
and efficient it is. Thus, it is not possible to use
a unique recognizer for all symbols a musical
score can contain. It is then necessary to exploit
dedicated recognizers , that is recognizers able to