Stakeholder Groups in Computational Creativity Research and Practice - Computational Creativity Research: Towards Creative Machines - page 18

The majority of researchers develop software using only themselves as an evalu-
ator, because observer-based models are too time-consuming to use on a day-to-day
basis. These informal in-house evaluation techniques generally do not capture the
global aims of the research project, or of the field (e.g., producing culturally impor-
tant artefacts and/or convincing people that software is acting in a creative fashion).
This can lead to situations where systems are presented as feats of engineering,
with little or no evaluation at all [ 9 ]. In [ 47 ], we argue that assessing progress is
inherently a process-based problem, and hence we concentrate our formalism on
processes, tempered with aspects of artefact evaluation. In the subsections below, we
present this formalism with worked examples, followed by a case study describing
the development of an evolutionary art system.
1.5.1 Formal Assessment of Progress
We combine the most useful aspects of the IDEA and FACE models, the list of
essential behaviours described in Sect. 1.4.1 , and certain aspects of assessing artefact
value in a diagrammatic formalism for evaluating progress in the building of creative
systems.We focus on the creative acts that software performs, the artefacts it produces
and the way in which audiences perceive it and consume its output. We simplify by
assuming a development model where a single person or team develops the software,
with various major points where the program is sufficiently different for comparisons
with previous versions. We aim for the formalism to be used on a daily basis without
audience evaluations, to determine short term progress, but for it also to enable fuller
audience-level evaluations at the major development points. We also aim for the
formalism to help determine progress in projectswhere there are both weak and strong
objectives, focused, respectively, on the production of increasingly higher valued
artefacts, and on increasing the perception of creativity people have of the system.
We found that the original FACE model didn't enable us to properly express the
process of building and executing generative software. Hence another consideration
for our formalism is that it can capture various timelines both in the development and
the running of software in such a way that it is fairly obvious where the programmer
contributed creatively and where the software did likewise.
1.5.2 Diagrammatic Capture of Timelines
Taking a realistic but abstracted viewof generative software development and deploy-
ment, we identify four types of timeline. Firstly, generative programs are developed
in system epochs , with new versions being regularly signed off. Secondly, each
process a program undertakes will have been implemented during a development
period where creative acts by programmer and program have interplayed. Thirdly,
at run-time, data will be passed from process to process in a series of creative and