Handle: Engineering Artificial Musical Creativity at the “Trickery” Level - Computational Creativity Research: Towards Creative Machines - page 295

Fig. 14.12 Two Frames from a Heider-Simmel-based video (a) Triangle “Examining” Box
(b) Triangle “Celebrating”

for story representation and film composing. The fact that the stories are natively in
a video format also helps with aligning music to the story. In addition, since there
is substantial consensus regarding what the stories are “about,” we can partially
measure how successful our music generation is by analyzing whether the music
enforces the perceived storyline or deprecates it. Lastly, by using the Heider-Simmel
stories we open up the opportunity for collaboration with other research teams (see
[ 25 ]). Figure 14.12 shows two frames from a Heider-Simmel based story we have
created.

14.5.3.2 Story Representation

In order for Handle film to generate music to match a given story, we need a represen-
tion scheme for stories that is in a form Handle film can use to reason about the story,
and to make decisions about what music is most appropriate. Unsurprisingly, we
have opted to use formal logic, starting with a proper fragment of the music calculus
discussed above; viz., the event calculus (EC), elegantly presented, in connection
with stories (specifically with the field of story understanding )in[ 18 ]. EC allows us
to represent a variety of simple stories and then reason about the events in them, and
about some basic emotional constructs related to those events. This information can
then be used to determine what kind of music is best suited for the story. Figure 14.13
shows the event-calculus representation of the story in Fig. 14.12 .
As the stories and our descriptions of them become more complex, EC is no
longer sufficient to represent all the narratological input that Handle film needs in
order to compose music to match the relevant story. For example, EC cannot properly
represent the beliefs and knowledge of the agents in the story. To represent these facets
of the story, we must supplement the EC by using the described previously (also

see [ 9 ]). Figure 14.13 shows our first steps in representing the Triangle's beliefs and
emotions via the following simple conditionals:

CEC

P ( Triangle , t , Fireworks ) ⃒ B ( Triangle , t , Celebration )

( , , ) ⃒ ( )

B Triangle t Celebration Happy Triangle