Environmental Engineering Reference
In-Depth Information
evolution of public policy, are very difficult to ''get right'' in
simulation and are perhaps best handled as a given, but external,
influence on system dynamics. Indeed, a model might actually
have been built to evaluate the nitty-gritty system dynamics
that follow on from the use of various policy ''levers'' on a
system's trajectory.
Growth rates may also be specified exogenously, for example,
when growth is assumed to originate outside a given urban
system, for example, through in-migration or as a function of
economic and demographic links with other systems (Torrens,
2006b). White and Engelen (1993), for example, treated growth as
external to their cellular automata models because they regarded
growth in a real city as being dependent on the position of a city
in its regional or national economy, rather than its internal spatial
structure. However, problems may arise if exogenously-specified
constraints conceal the actual interactions between local states in
a CA (Straatman, White and Engelen, 2004).
make random choices between similarly-optimal adjustments, a
variety of techniques were proposed, including making the length
and width searches inter-dependent, varying those approaches,
and making use of ''backtracking'' (setting the adjustment pro-
cedure back by a number of transition steps, and relaxing the
adjustment to allow for less-than-optimal solutions).
An alternative method for automatic calibration involves the
use of self-modifying mechanisms: transition rules are allowed
to change (in relative importance or weighting) as a simulation
evolves. Under self-modifying calibration schemes, changes in
parameter values are often linked to evolving conditions within
the model itself. In the SLEUTH model, for example, the rate of
growth serves as a trigger for adaptation in the application of rules
(Clarke, 1997; Clarke, Hoppen and Gaydos, 1997; Clarke and
Gaydos, 1998; Candau, Rasmussen and Clarke, 2000; Silva and
Clarke, 2002; Goldstein
et al
., 2004; ). Under conditions of rapid
growth, growth control parameters in the model are exaggerated.
Essentially, this acts as a brake on the growth metabolism or
momentum of the model. If a simulation exhibits ''little or
no growth'', the growth parameters are adjusted to dampen
growth. Without self-modification of this nature, simulation
runs would produce only linear or exponential growth (Silva and
Clarke, 2002).
23.2.6
Automatic calibration
Urban automata models may be specified with a bewildering
array of inter-dependent parameters and widgets. Calibrating
those mechanisms often requires the use of high-performance
computing (Hecker
et al
., 1999; Bandini, Mauri and Serra, 2001;
Nagel and Rickert, 2001; Guan, Clarke and Zhang, 2006). Sev-
eral authors have developed schemes for automatic calibration,
typically adding calibration modularly as a step in a simulation
run. A range of techniques have been employed to achieve this,
including brute-force approaches, optimized search routines,
and self-modification schemes.
Brute-force calibration (Silva and Clarke, 2002) is a procedure
for essentially throwing computer power at a model to run it
over many permutations and combinations of parameter values.
Several developments to the SLEUTH model have been under-
taken around these goals (Guan, Clarke and Zhang, 2006). In
such instances, results of varying parameters are sorted according
to some metric, and the highest-scoring results are fed into the
next iteration of the procedure.
Optimized search procedures are similar to the brute-force
approach, but introduce an element of machine intelligence.
Unlike the brute-force approach, which rank-selects parameter
values rather simply, optimized search schemes use a variety
of guidelines (decision rules) to target parameter adjustments.
This approach begets related issues: how to gauge error; how
to formulate a decision rule (adjust up, adjust down, aim for
the optimal solution, go for a less-than-optimal solution); how
to choose between adjustments that appear to yield the same
improvement; and what to do if the adjustment decision yields
an unsatisfactory result that manifests only after several subse-
quent transitions have taken place. In the example developed by
Straatman and colleagues (2004), a procedure was introduced
that targeted searches based on benchmarks for maximum error
(the neighborhood with the greatest difference between desired
potential and undesired potential) and total error (the number
of neighborhoods, or cell count, that resulted in a wrong state).
Adjustment was carried out by means of ''length searches''
(adjustments that yield a result that is more likely to convene on
a desired state, or less likely to result in an undesired state) and
''width searches'' (adjustment guided by the relative ability of
the adjustment options to reduce total error). To avoid having to
23.3
Validating automata
models
For the most part, model calibration takes place before a sim-
ulation run. Validation relates to the assessment of a model's
performance, and this generally takes place after a simulation has
been run. In abstract terms, we may differentiate between qual-
itative validation and quantitative validation. The former refers
to the evaluation of general agreement between a simulation and
observed conditions; the latter denotes the assessment of empir-
ical goodness-of-fit between simulation outputs and observed
conditions. We can also distinguish between validation mecha-
nisms designed to assess model performance through analysis of
the patterns and outputs that a simulation generates and mecha-
nisms that analyze a simulation run itself, as a simulation of the
system being considered.
23.3.1
Pixel matching
Validation by visual inspection is one of the most straightforward
(but subjective) techniques for assessing the performance of a
model. In a sense, model designers or users act as their own
test case in studying the system as it unfolds in simulation. For
example, one might evaluate whether a model generated plausible
urban forms (Wu, 1998), or whether the simulated processes
operated at sensible rates and with appropriate consequences
(Torrens, 2006b). In their models of urban growth in the San
Francisco Bay Area, Clarke and colleagues used visual validation
to determine estimates for parameter settings in the model.
Simulations were run, their performance was evaluated visually,
and parameters were adjusted based on those evaluations if
necessary (Clarke, Hoppen and Gaydos, 1997). They looked, in
particular, at whether their model generated realistic patterns of
Search WWH ::
Custom Search