Information Technology Reference
In-Depth Information
Common practices such as verification and validation are well-known quality
control procedures for assessing the quality of scientific models in general (Cover
and Curd, 1998). However, verification and validation are insucient criteria for
assessing the quality of social science models, specifically for social simulations.
An important implication is that current emphasis on model verification and
validation is warranted (Cio, 2010; Sargent, 2004), but
verification and valida-
tion are insu
cient by themselves for judging the quality of a social simulation
model
(agent-based or other).
Therefore, a key methodological question concerning quality is: Which addi-
tional criteria—i.e., beyond Truth, Beauty, and Justice—could or should be used
to assess the quality of a social simulation model? The next section addresses
this question by proposing a set of dimensions for evaluating the quality of a
given social simulation model.
2 Dimensions of Quality in Social Simulation Models
The quality of any complex artifact—whether a social simulation model or the
International Space Station—is a multifaceted property, not a single dimension.
Dimensions of quality can be used for evaluation as well as for a checklist of desir-
able attributes for building and developing a social simulation model. Arguably,
there are two levels of quality assessment for computational social simulations,
corresponding to the concepts of
amodel
and
modeling
, respectively.
First, from a model's perspective, any set of quality dimensions for evalu-
ating a social simulation must be based on its specific attributes or uniquely
constituent features as a computational artifact in the sense of Simon (1996).
Moreover, whether the overall quality of a given model should be an additive
or a multiplicative function of individual qualitative features is less important
than the idea that overall quality depends on a set of dimensions or desirable
features beyond the Lave-March criteria, not on some single preeminent feature
(e.g., simulation environment or programming language).
Second, from a modeling perspective, quality assessment should cover the
broader modeling or model-building process as such, beyond the social simulation
model that is produced in a narrow sense. This is because a computational model
in final (i.e., committed) instantiated code is the result of a sequence of earlier
modeling stages that precede the model itself, such as the critical stage of model
design prior to implementation. Quality in design affects quality in the product
of implementation, even when implementation
per se
is carried out in a proper
manner (i.e., competently, with effectiveness and eciency).
The following framework for quality assessment combines both perspectives
by focusing on the classical methodological stages of social simulation model
development:
1. Formulation
2. Implementation
3. Verification
4. Validation
