Information Technology Reference
In-Depth Information
various descriptors of the spectrum shape such
as the Spectral Centroid and Bandwidth. Another
widely used frequency domain spectrum summa-
rization is Mel-Frequency Cepstral Coefficients
(MFCCs) which originated from speech and
speaker recognition. The goal of time domain
summarization is to characterize the musical
signal at a longer time scale than the short-time
analysis slices. Typically this summarization is
performed across so called “texture” windows
of approximately 2-3 seconds or it can be also
performed over the entire piece of music. Figure
2 shows graphically frequency and time summa-
rization. Later in this chapter we show how this
process can be represented in
MARSYAS
.
MARSYAS 0.1
mainly supported audio
analysis. One of the main motivations behind the
redesign/rewrite of version 0.2 was the desire to
also support audio synthesis. Even though not
immediately obvious, audio synthesis can be
very useful in MIR systems and applications. For
example while researching an algorithm for drum
transcription it is very useful to be able to hear a
resynthesized sound contains only drum sounds.
Listening to the result can reveal information that
maybe hard to obtain from plots or just numeric
results. Similarly synthesizing the result of pitch
extraction can be very informative about the
nature of pitch errors. Although it is possible to
use a variety of different tools for these purposes,
having them all integrated under one system can
be very helpful.
The central challenge in the design of an audio
processing framework is the need for very efficient
runtime performance while retaining expressivity.
Frequently audio researchers are faced with a hard
dilemma. They can use interpreted programming
environments such as MATLAB that provide a lot
of necessary components but are not as efficient as
compiled code or write code from scratch which
requires a significant investment of time just to
build the necessary infrastructure.
MARSYAS
at-
tempts to balance these two extremes. As will be
described later in this chapter, a lot of flexibility
and functionality is provided at run-time. For
example, the user can easily create complicated
networks of processing objects and control them
without recompiling code. However the resulting
system is still very efficient as it relies on compiled
code. The only time that code recompilation is
required is when new processing objects need to
be written.
Two other important requirements for a suc-
cessful audio framework are interoperability
and portability. Most researchers utilize a large
ecology of different tools to accomplish their
task. To be useful, a framework must provide
ways to communicate with other tools. For ex-
ample
MARSYAS
provides a variety of tools for
communicating with specific programs such as
WEKA (for machine learning) and MATLAB (for
numerical calculation) as well as general ways for
communication with graphical user interfaces.
Portability is also very important as operating
systems and hardware change all the time. For
example, initially
MARSYAS
was developed on
SGI machines running IRIX which is not sup-
ported any more, but OS X which did not exist
at the time is supported now.
These requirements make the development
of audio processing frameworks challenging.
However, there are some characteristics of audio
that can help the designer. Audio processing has
a strong notion of time. There is a constant flow
of data through the system. In most cases there
is little need for complicated dependencies be-
tween processing chunks of data. Therefore, for
a specific application it is relatively easy to create
efficient code with a fixed memory footprint. With
a little bit more work it is possible to generalize
the process so that most audio applications can
be expressed easily without sacrificing run-time
performance.
In the following sections we describe the
general architecture of
MARSYAS
and how these
requirements are addressed by specific design
decisions and concepts. We believe that many of
Search WWH ::
Custom Search