Information Technology Reference
In-Depth Information
these concerns and their solutions are relevant not
only to programmers and users of
MARSYAS
but
developers of any MIR software system.
In
MARSYAS
terminology the processing nodes
of the dataflow network are called
MarSystems
and provide the basic components out of which
more complicated networks are constructed.
Essentially any audio processing algorithm can
be expressed as a large composite
MarSystem
,
which is assembled by appropriately connected
basic
MarSystems.
Some representative
MarSystems
provided
are the following:
Architecture
Dataflow programming is based on the idea of
expressing computation as a network of process-
ing nodes/components connected by a number
arcs/communication channels. Computer Music
is possibly one of the most successful application
areas for the dataflow-programming paradigm.
The origins of this idea can possibly be traced
to the physical rewiring (patching) employed for
changing sound characteristics in early modular
analog synthesizers.
Expressing audio processing systems as data-
flow networks has several advantages. The pro-
grammer can provide a declarative specification
of what needs to be computed without having to
worry about the low-level implementation details
of how it is computed. The resulting code can be
very efficient and have a small memory footprint
as data just “flows” through the network without
having complicated dependencies. In addition,
dataflow approaches are particularly suited for
visual programming. One of the initial motiva-
tions for dataflow ideas was the exploitation of
parallel hardware and therefore dataflow systems
are particularly good for parallel and distributed
computation.
The main goal of
MARSYAS
is to provide a
general, extensible and flexible framework that
enables experimentation with algorithms and pro-
vides the fast performance necessary for develop-
ing real-time audio analysis and synthesis tools. A
variety of existing building blocks that form the
basis of many published algorithms are provided
as dataflow components that can be composed to
form more complicated algorithms (black-box
functionality). In addition, it is straightforward to
extend the framework with new building blocks
(white-box functionality).
•
Input/Output (Sources and Sinks)
o
SoundFile I/O for .wav, .au, .mp3 and
.ogg files
o
Real-time audio I/O using RtAudio
o
Matlab, Weka, Octave I/O
•
Feature Extraction
o
Shor t-Ti me Fou r ie r Tr a n sfor m
(STFT)
o
D i s c r e t e -Wa v e l e t
Tr a n s f o r m
(DWT)
o
Centroid, Rolloff, Flux, Contrast
o
Mel-frequency Cepstral Coefficients
(MFCC)
o
Auditory Filterbanks
•
Synthesis
o
Wavetable synthesis
o
FM synthesis
o
Phasevocoder
•
Machine Learning
o
Gaussian Mixture Models (GMM)
o
K-Nearest Neighbor
o
Principal Component Analysis
o
K-Means Clustering
o
Support Vector Machine (SVM)
In addition to being able to process data,
MarSystems
need additional information that can
change their functionality while they are running
(processing data). For example a
SoundFileSource
needs the name of the sound file to be opened, and a
Gain
can be adjusted while data is flowing through.
This is achieved by a separate message passing
Search WWH ::
Custom Search