Information Technology Reference
In-Depth Information
Convergence Classification and Replication Prediction
for Simulation Studies
Andreas D. Lattner
1
, Tjorben Bogon
1
,
2
, and Ingo J. Timm
2
1
Information Systems and Simulation, Institute of Computer Science
Goethe University Frankfurt, P.O. Box 11 19 32, 60054 Frankfurt, Germany
2
Business Informatics I, University of Trier, D-54286 Trier, Germany
Abstract.
Providing assistance systems for simulation studies can support the
user by performing monotonous tasks and keeping track of relevant results. In
this paper we present approaches to estimate, if - and when - statistically sig-
nificant results are expected for certain investigations. This information can be
used to control simulation runs or to provide information to the user for interac-
tion. The first approach is used to classify if significance is expected to occur for
given samples and the second approach estimates the needed replications until
significance is expected be reached. For an initial evaluation of the approaches,
experiments are performed on samples drawn from normal distributions.
Keywords:
Significance
estimation,
Simulation
control,
Statistical
tests,
Machine learning.
1
Introduction
Nowadays, simulation is widely used in order to evaluate system changes, to perform
parameter optimization of systems, or to compare existing alternatives. A clear advan-
tage of simulation is that costs or damages on real systems can be avoided while in-
vestigating effects of changes or testing newly planned systems. Simulation is used in
various domains, e.g., for marine container terminal planning (Berth Planning and Quay
Resources Assignment Problem; [10]), multi-location transshipment problems [3], and
clinical resource planning [15].
If complex systems with many parameters are modeled, simulation studies can con-
sist of a large number of single simulation runs and a rather structured and disciplined
evaluation has to be performed in order to avoid getting lost in the vast of result data. A
support for the non-creative, monotonous tasks in simulation is desirable.
In this work, we present one aspect of the current research project
AssistSim
address-
ing a support for the performance of simulation studies. The project aims at supporting
planning and execution of simulation studies including simulation system control and
an automated analysis of intermediate simulation results. In this paper we present an
approach to significance estimation in order to estimate, if - and when - statistically
significant results are expected for certain investigations. The approach itself can also
be applied in other situations, i.e., beyond simulation - for any task where two samples
should be compared and where preliminary samples should be used for estimation how
many further examples might be needed in order to satisfy certain statistical properties.
