Information Technology Reference
In-Depth Information
section provide a viable framework while social simulation develops as a field,
not as a permanent set of fixed criteria. Each stage in the Lifecycle Framework
contains numerous dimensions for quality assessment because social simulations
are complex artifacts, in the sense of Simon.
Interestingly, Osgood's first dimension in cognitive EPA-space is Good-Bad
(evaluation). This is why quality evaluation (good-bad-ugly) is essential (Os-
good, May, and Miron, 1975). The proposed criteria should allow a classification
of social simulations into categories of good, bad, or outright ugly.
As computational social scientists we need to better understand the micro-
processes that compose the overall quality of social simulation:
-
How is a problem chosen for investigation?
-
How is the problem-space reduced by abstraction?
-
How is the model designed?
-
How well are the entities and relations understood?
-
How is the simulation language chosen?
-
How is the model implemented?
-
How are verification and validation conducted?
-
How are simulation runs actually conducted?
-
How is the model being maintained?
Requiring additional quality criteria for social simulation models is not an ar-
gument against the unity of science. It is a plea for greater specificity and more
rigor in the evaluation of quality in the field of social simulation.
From a methodological perspective, quality criteria could also help support
the (virtual) experimental function of social simulations in terms providing ways
for assessing the veracity of artificial worlds. Computational experiments could
thus be framed within the context of a social simulation characterized by a set
of quality features, taking all lifecycle stages into consideration, as in evaluating
experimental results
conditional upon
the quality of the social simulation model.
Such a function would enhance the value of social simulation as an experimental
method and highlight its scientific usefulness.
Finally, the topic of quality in social simulations also motivates a broader
discussion concerning similarities and differences between social simulation and
other scientific approaches in science, such as statistical and mathematical mod-
els. While all scientific approaches share some of the same quality criteria, each
has also unique quality criteria that are not applicable in other approaches.
For social simulations there are aspects such as quality of code or visualization
dashboards that are
sui generis
to the approach itself. The lifecycle approach
to assessing quality in social simulations could also shed new light on parallel
efforts in statistical and mathematical models of social systems, since there too
we find a similar sequence of stages, from the formulation of research questions
to analyzing and communicating model results, albeit with significant variations
in technical details if not in the overall process.
