Information Technology Reference
In-Depth Information
We are working on incorporating more sophis-
ticated techniques for finding hints. Instead of
using an overall multiple of the beat of the song,
we wish to find where the song changes time or
introduces an interlude or a break and have the
agent direct the visual performance accordingly.
Additionally, we will introduce work on song
structure identification to find such things as
verse/chorus boundaries (Cooper & Foote, 2003)
in the song to provide further direction. MusicS-
tory could perform a call back to a prior image
during a later chorus in the song.
Currently, we look for features present in 20
and 30 something pop/rock. MusicStory could be
expanded to more demographics by first using
genre classification. Genre knowledge can be used
to identify which features and metrics to look
for within the song. We also plan to incorporate
our existing work on affect detection to steer the
direction of the music video. High affect lyrics
can be matched to similar high affect images,
which can be found from reading the affect in
the image's captions, tags, and comments as well
as image analysis.
MusicStory is what is known as a “mashup,”
an application which combines media and content
from several sources to make a new application.
Flickr itself has an entire gallery of mashups
submitted by third parties. Since MusicStory
(and the Imagination Environment), mashup for
storytelling through appropriated images has
appeared in several installations. For instance,
September 23, 2006 brought
StoryMashup
to
the streets of New York City. Two hundred and
fifty participants of this game, covered midtown
Manhattan with Nokia camera phones. Each
phone was sent a stream of keywords. Partici-
pants were scored with how quickly they could
take a representative photo of words like “taxi,”
“fire,” and “harmony.” The game continued for
90 minutes. When the game was over, the photos
served as illustrated images to stories presented
on the Reuters billboard in Times Square (Tuulos
& Scheible, 2006).
New directions in automated content creation
incorporate new media content from our social
spaces. As automatic music video creation moves
forward, it will include new techniques in video
remixing and location based content creation.
These directions must be approached paying
careful attention to the domain semantics, rely-
ing on techniques described via filmmakers, and
hybrid approaches which utilize content found
autonomously.
acknoWledgment
The authors thank Victor Friedberg and Melanie
Cornwell from NextFest & Wired Magazine,
Jeff Tweedy from the band Wilco, and Flickr. In
addition, continuing thanks to the guiding com-
ments of Kristian Hammond, Larry Birnbaum,
and other members of the Intelligent Information
Laboratory at Northwestern University.
references
Anderson, C. (Ed.). (2005, June). NextFest.2005.
Wired Magazine, 13
(6), p. 24.
Berenzweig, A., & Ellis, D. (2001). Locating
singing voice segments within music signals. In
Proceedings of the IEEE Workshop on Applica-
tions of Signal Processing to Audio and Acoustics,
New York.
Budzik, J. (2003, June).
Information access in
context: Experiences with the Watson System
.
Unpublished doctoral
dissertation. Evanston, IL:
Northwestern University.
Cooper, M., & Foote, J. (2003). Summarizing
popular music via structural similarity analysis.
In
Proceedings of the IEEE Workshop on Ap-
Search WWH ::
Custom Search