Geology Reference
In-Depth Information
valid proofs. Huang [ 33 ] made some recommendations for the two-hidden-layer
case. He suggested the number of hidden nodes suf
cient to train N samples with a
reasonable minimum error is
p
ð
N hid ¼
2
M
þ
2
Þ
N
ð 2 : 3 Þ
The suf
cient number of hidden nodes in the
first layer is
s
N
p
ð
N1 hid ¼
2
M
þ
2
Þ
N
þ
2
ð 2 : 4 Þ
ð
M
þ
2
Þ
The suf
cient number of hidden nodes in the second layer is
s
N
N2 hid ¼
M
ð 2 : 5 Þ
ð
M
þ
2
Þ
In all these equations, M = output neurons and N = training data points.
Stathakis [ 72 ] suggests the most accurate structure will have fewer nodes than
the one suggested by Huang [ 33 ] and this high structure leads to redundancy in
structure and over-
tting of the training data. Stathakis [ 72 ] proposed a near-
optimal solution approach with a GA to
find a better topology.
cation of topology has been based on trial and error, on
heuristic approaches, on heuristics sometimes followed by trial and error, and on
pruning or constructive methods.
Trial and error: This is the most traditional and primitive way of assessment. It
may yield severely suboptimal structures, especially when adopted by inexperi-
enced users.
Heuristic methods: Several approaches are found in the literature [ 8 , 59 , 79 ]
which are all based on the objective to devise a formula which estimates the number
of nodes in the hidden layers as a function of the number of input and output nodes.
However, most of the heuristics lack the theoretical evidence to support the dis-
covery of an optimal structure, so they are commonly used in subsequent search by
trial and error.
Exhaustive search: This is one of the perfect but impracticable approaches in real
life applications, as the number of search alternatives is exceedingly large, com-
putationally intensive, and with longer computation time. Yao [ 82 ] illustrates the
dif
Traditionally, identi
culties in exhaustive searching to
find hidden neurons; he identi
ed that the
major complication is due to the noisy
fitness evaluation problem.
Pruning and constructive algorithms: These are developed with the objective of
devising an effective network topology by incrementally adding or removing links
(weights) to the redundant or simple structures, respectively. Optimal Brain
Damage [ 44 ] and Optimal Brain Surgeon [ 29 ] are two commonly used algorithms.
 
Search WWH ::




Custom Search