Digital Signal Processing Reference
In-Depth Information
to structurally similar neurons. This is not the case with RBF networks, for
which the nature of the centers and variances are distinct from that of the
output weights. This certainly does not mean that the entire set of parame-
ters cannot be jointly adapted [140]. However, this procedure will be related
to a complex MSE surface with a removable potential for multimodality.
So, it is interesting to consider the option of separating both adaptation
processes.
By carrying out a separated optimization, if we suppose the hidden layer
to have been properly designed, the remaining problem of finding the out-
put weights will be linear with respect to the free parameters. Thus, we will
be back within the framework of linear supervised filtering, and it will be
possible to obtain a closed-form solution or to make use of algorithms like
LMS and RLS (vide Chapter 3). Since a given choice of the centers and vari-
ances potentially establishes the optimal solution of the output weights, we
are led to the conclusion that the crux of this approach is the project of the
hidden layer.
There are many possibilities for carrying out center placement, for
instance, possibilities that range from a uniform distribution to the use of
some clustering process [44]. If the idea is to use a clustering method, a clas-
sical option is to employ the
k
-means algorithm [105], the summary of which
is given in Algorithm 7.3.
Algorithm 7.3
:
k
-Means Algorithm
1. Initialize the centers
2. While the stopping criterion is not met
(a) For
n
1to
N
samples
(i) Determine the distance between each center and the input
vector
x
=
.
(ii) Assume that the center closest to the input pattern is labeled
as the
k
th center. Update it according to the following rule:
(
n
)
c
k
←
c
k
+
[
(
n
)
−
c
k
]
μ
x
(7.50)
(iii) End for
3. Randomly rearrange the data set.
4. End while
The
k
-means algorithm [105] is founded on a process that can be
described as competitive learning [140], in which the centers compete for the
right of representing the data. The winner is updated in a manner that tends
to reinforce the suitableness of its representation, the outcome of the method
