Geology Reference
In-Depth Information
2.5.2 Model Complexity
In the last 20 years, the study of complexity in modeling systems has emerged as a
recognized
field in statistics. However, the initial attempts to formalize the concept
of complexity go back even further, to Shannon
'
s inception of Information theory
[
66
]. The complexity of a model is closely related to the uncertainty of the system,
which can be de
ned in terms of model properties such as model sensitivity and
modeling error. The general hypothesis of model complexity and its influence
during training and testing phases is shown in Fig.
2.3
. The general hypothesis
states that more complex models can simulate reality better than simpler models
(i.e., less prediction error), and with a greater variance and low bias during training
phase. Less complex models provide a relatively approximate simulation (i.e., with
more prediction error), but with less variance and higher bias. However, the case is
somewhat different in the testing phase; highly complex models won
t give the best
test results as the graph is parabolic, with a minimum somewhere in the middle.
Figure
2.4
displays the hypothesis which shows the variation of different model
parameters, particularly with bias-variance interaction during the test phase.
'
Fig. 2.3 Hypothesis showing
the effect of complexity
during training and testing
[
30
]
Fig. 2.4 Hypothesis showing
effect of complexity on bias-
variance interaction
