Information Technology Reference
In-Depth Information
Chapter XIII
MusicStory:
An Autonomous, Personalized
Music Video Creator
David A. Shamma
Yahoo! Research Berkeley, USA
John Woodruff
Ohio State University, USA
Bryan Pardo
Northwestern University, USA
aBstract
This chapter covers some of the challenges in storytelling with music. We describe the MusicStory system
which creates music videos from personal media (audio and image) collections. Using a song's lyrics,
MusicStory finds and presents word/image associations. It takes the emotional experience of listening
to music, amplifies it and heightens its visceral appeal by externalizing concrete and visual imagery
intrinsic in the music. The retrieved images vary in their association—some semantically on point and
some distant. The flow of imagery moves with the pace of the song: providing quick transitions through
fast songs, and leisurely transitions through slower songs. In addition, MusicStory uses vocal segmen-
tation to direct the video and alter how images are displayed. We discuss the creative and technical
challenges in this work as well as how it was deployed in several fashions.
IntroductIon
increasingly stored online. Visual images can be
found on such photo sharing Web sites as Flickr.
Music sites, such as iTunes, eMusic, and Rhapsody
make large collections of music audio available
on the internet. Lyrics are available at sites such
as Leos Lyrics. This provides unprecedented
opportunity to explore the relationships between
imagery, audio, and text documents through the
use of autonomous information systems.
Our personal media collections, from the music on
an iPod to the images in a photo album, reflect who
we are as individuals. Public media collections,
from commercial image repositories, to the songs
in a band's MySpace account, reflect who we are
as a culture. Today, these media collections are
Search WWH ::
Custom Search