Handle: Engineering Artificial Musical Creativity at the “Trickery” Level - Computational Creativity Research: Towards Creative Machines - page 294

Fig. 14.11 Process for creating music from narrative
for the use of visual stimuli in the composition of music. (For full information, see
[ 27 ].) Most importantly for Handle jazz , however, it offers a resource to manage and
easily explore the large data-space resulting from the many possible combinations
of musical features captured hitherto, as well as any we hope to capture in the future.
14.5.3 Toward Handle as Film Composer
Handle film reflects a turn toward a new component of the creative process: com-
posing music for film. Our starting point in this endeavor parallels the starting point
of film music as a whole. When silent films first started using music, there was a
pianist or organist who would play music along with the video. This music was often
improvised, but for those who were not as comfortable with improvisation there were
topics of music printed with basic instructions indicating what music was appropriate
for many different types of scenes; initially, we take a similar approach [ 24 ]. Taking
a story in the form of video, we represent it in a form that Handle film can reason over
and gather knowledge about. Then, using this knowledge, Handle film can select a
piece of music from a predefined database of music that contains both a list of the
pieces available and a description of when each piece is most suitable. Once a piece
of music is selected, it is then matched to the video. The entire high-level process
can be seen in Fig. 14.11 .
14.5.3.1 Story Foundation
In 1944, Fritz Heider and Marianne Simmel carried out a seminal and now-famous
experiment to study human perception of the behavior of others. The experiment used
videos portraying abstract shapes—triangles and circles to be exact—interacting in
an ambiguous but highly suggestive manner [ 16 ]. Heider and Simmel found that
most people watching the video assign personalities to the shapes and a storyline
to the video to explain the interactions between these shapes; they also found that
people invent similar personalities and storylines. Handle film is currently based on
stories we invent for a type of action seen in this original experiment. These stories
are relatively short, but are still interesting this makes them a good starting point