Geology Reference
In-Depth Information
is to present a large number of inputs to the model and rely on the network to
identify the critical model inputs. Usually, not all of the available data pool will be
equally supportive for effective modeling, since some may be redundant with very
close correlation with another; some may have predominant noise over the infor-
mation or may not have any appreciable relationship with the target variable of the
expected study. So such a practice normally causes adverse effects on modeling
results. Another serious issue is that the redundancy of a network is related to both
the number of weights and the size of the weights. Selection of appropriate and
effective models connects with the number of weights, and hence the number of
hidden units and layers. Until now there has been no exact solution for questions
such as how many hidden layers and how many hidden nodes there should be in
node-based modeling [
51
,
72
]. The selection of hidden neurons is the tricky part in
ANN modeling, as it relates to the complexity of the system being modeled and is
usually set by the user. There should be an effective way to decide on the number of
nodes, considering many factors such as number of input and output units, number
of training data points, amount of noise in the targets, complexity of the function or
classi
cation or learning algorithm, topology of the model, type of hidden unit
activation function, and regularization.
There are many practical rules of thumb that are available in the literature to
facilitate a decision on the number of hidden nodes. Blum [
17
] reports that the
number of nodes in the hidden layer
—
somewhere between the input layer nodes
and the output layer node size
is appropriate for modeling. Hecht-Nielsen [
32
]
proposes that the maximum number of elements in the hidden layer be twice the
input layer dimension plus one. Another study by Maren et al. [
50
] recommends
using the number of nodes equal to the geometric average between the input and
output node dimension. Mechaqrane and Zouak [
52
] have used a feed-forward
network with the size of the hidden layer equal to the size of the input layer. Some
companies working on commercial neural network software development adopt a
rule of thumb of the sum of input and output nodes multiplied by 2/3 as the
indicator to choose the number of hidden neurons. Swingler [
73
] suggests that, for
networks with one hidden layer, the model give better performance if we use twice
the number of input nodes in the hidden layer. At the same time, Berry and Linoff
[
13
] note that the number of hidden nodes should never be more than double the
nodes in the input layer. Boger and Guterman [
18
] used principle component
analysis to
—
find the number of hidden nodes and they suggested using the same
number of components which express 70
90 % of the variance of the input data.
However, our experience shows that, in general, these rules give only some indi-
cation for the hidden layer dimension and none are properly right or wrong.
The above-mentioned Hecht-Nielsen suggestion has more scienti
-
c authenticity
as the method is based on the Kolmogorov theorem in node based computation.
There were strong arguments against this recommendation by many researches
[
33
,
34
], saying that it is suf
cient to use a single hidden layer when using regular
transfer functions (e.g., sigmoidal) but the number of required hidden nodes can be
as high as the number of training samples, and justifying their arguments through
