Databases Reference
In-Depth Information
3.4.1.
GMDH-type polynomial neural network model
The GMDH belongs to the category of inductive self-organization data
driven approaches. It requires small data samples and is able to optimize
models' structure objectively. Relationship between input-output variables
can be approximated by Volterra functional series, the discrete form of
which is Kolmogorov-Gabor Polynomial
107
:
y
=
C
0
+
k
C
k
1
x
k
1
+
k
C
k
1
k
2
x
k
1
x
k
2
+
k
C
k
1
k
2
k
3
x
k
1
x
k
2
x
k
3
(3.4)
1
1
k
2
1
k
2
k
3
where
C
k
denotes the coecients or weights of the Kolmorgorov-Gabor
polynomial and
x
vector is the input variables. This polynomial can
approximate any stationary random sequence of observations and it can
be solved by either adaptive methods or by Gaussian Normal equations.
This polynomial is not computationally suitable if the number of input
variables increase and there are missing observations in input dataset. Also
it takes more computation time to solve all necessary normal equations
when the input variables are large.
A new algorithm called GMDH is developed by Ivakhnenko
108,109
which
is a form of Kolmogorov-Gabor polynomial. He proved that a second order
polynomial i.e.:
y
=
a
0
+
a
1
x
i
+
a
2
x
j
+
a
3
x
i
x
j
+
a
4
x
i
+
a
5
x
j
(3.5)
which takes only two input variables at a time and can reconstruct the
complete Kolmogorov-Gabor polynomial through an iterative procedure.
The GMDH algorithm has the ability to trace all input-output relationship
through an entire system that is too complex. The GMDH-type Polynomial
Neural Networks are multilayered model consisting of the neurons/active
units/Partial Descriptions (PDs) whose transfer function is a short term
polynomial described in equation (3.5). At the first layer
L
=1,an
algorithm, using all possible combinations by two from
m
inputs variables,
generates the first population of PDs. Total number of PDs in first layer
is
n
=
m
(
m
1)
/
2. The output of each PD in layer
L
= 1 is computed
by applying the equation (3.5). Let the outputs of first layer be denoted as
y
1
,y
2
,...,y
n
.The vector of coecients of the PDs are determined by least
square estimation approach.
Depending on the growth of the PNN layers the number of PDs
in each layer also grows, which requires pruning of PDs so as to avoid
the computational complexity. From the experimental studies it has been
−
