Information Technology Reference
In-Depth Information
6.
Dissemination.
Finally, the quality of a social simulation should be as-
sessed in terms of its “life-beyond-the-lab.” For instance, in terms of peda-
gogical value: Does the model teach well? I.e., does it teach eciently and
effectively? In terms of communicative clarity and transparency: Are useful
flowcharts and UML diagrams of various kinds (class, sequence, state, use
case) provided for understanding the model? Are they drawn with graphic
precision and proper style (Ambler, 2005)?In terms of replicability, what is
the model's replication potential or feasibility? How is reproducibility facili-
tated? Aspects related to a model's graphics are also significant for assessing
quality, not just as “eye candy.” In terms of GUI functionality, is the user
interface of high quality according to the main users? Is the GUI fundational
for answering the research questions? More specifically, in terms of visual-
ization analytics: Is visualization implemented according to high standards
(Thomas and Cook, 2005)? This does not concern only visual quality (Tufte,
1990), but analytics for drawing valid inferences as well (Cleveland, 1993;
Few, 2006; Rosenberg and Grafton, 2010). In terms of “long-term care:”
What is the quality of the model in terms of curatorial sustainability? How
well is the model supported in terms of being easily available or accessi-
ble from a long-term perspective? In which venue (Google Code, Source-
forge, OpenABM, Harvard-MIT Data Center/Dataverse, or documentation
archives such as the Social Science Research Network SSRN) is the model
code and supplementary documentation made available? Finally, some so-
cial simulations are intended as policy analysis tools. Is the model properly
accredited for use as a policy analysis tool, given the organizational mis-
sion and operational needs of the policy unit? Does the model add value to
the overall quality of policy analysis? Does it provide new actionable infor-
mation (new insights, plausible explanations, projections, margins of error,
estimates, Bayesian updates) that may be useful to decision-makers?
3 Discussion
Verification and validation are obviously essential dimensions of quality in social
simulation models; but they are just two among other dimensions of interest in
assessing quality across the full spectrum of model development stages. Thus,
and contrary to common belief, verification and validation may be viewed as
necessary but insucient conditions of a high-quality social simulation. The
many dimensions of quality in social simulations extend far beyond and much
deeper than the Lave-March TBJ criteria. Some of these criteria may be viewed
as utilitarian (e.g., based on resources), while others are non-utilitarian (based
on style).
Quality has dimensions because it is a latent concept, rather than a single
directly measurable property. Therefore, proxies (i.e., measurable dimensions
or attributes) are needed. The quality dimensions proposed in the preceding
