Information Technology Reference
In-Depth Information
the error objective. Also, many suboptimal functions may yield good results.
For example, in the following experiments, a typical approximation premise that
assumes a Gaussian distribution for the inputs has been implemented, proposing
the function for weight updating (known as a
weighting function
)[1],given
by Eq. (6).
To analytically introduce AMP in an arbitrary MLP training, all that has to
be done is to introduce the weighting function in the error function between the
real and the desired network output, as a function of the weights matrix
W
(
t
)
in each training iteration,
t
,thatis
E
[
W
(
t
)]
E
∗
[
W
(
t
)] =
(10)
f
X
(
x
)
And apply the BPA [15] to the weighted error
E
∗
(
W
)
for weights reinforcement
in each iteration
t
∈
N
,
i
∈
N
are the MLP layer, node and input counters
.If
s
,
j
component,
w
(
s
)
ij
R
+
the
respectively, for each
W
(
t
)
(
t
)
∈
R
,andbeing
η
∈
learning rate
, then the weight reinforcement in each iteration is given by:
η
∂E
∗
[
W
(
t
)]
w
(
s
)
ij
w
(
s
)
ij
(
t
+1)=
(
t
)
−
∂w
(
s
)
ij
η
f
X
∂E
[
W
(
t
)]
w
(
s
)
ij
=
(
t
)
−
(11)
∂w
(
s
)
ij
So, as the pdf weighting function proposed is the distribution of the input pat-
terns that does not depend on the network parameters, the AMMLP algorithm
can then be summarized as a weighting operation for updating each weight in
each MLP learning iteration as:
Δ
∗
w
w
∗
(
=
x
)
Δw
(12)
being
Δw
=
w
(
t
+
l
)
−
w
(
t
)
the weight updating value obtained by usual BPA
and
w
∗
(
the realization of the described weighting function
w
∗
(
x
)
x
)
for each
input training pattern
. During the training phase, the artificial metaplasticity
multilayer perceptron could be considered a new probabilistic version of the
presynaptic rule, as during the training phase the algorithm assigns higher values
for updating the weights in the less probable activations than in the ones with
higher probability.
x
6 Discussion
Plasticity and Metaplasticy still pose a considerable challenge for research in
terms of experimental design and interpretation [22]. Along the years, differ-
ent mathematical models of synaptic computation have been proposed. In the
classical Hebb model [11], the curve relating the increment of synaptic weight
to postsynaptic activation is a straight line without synaptic depression. In Se-
jnowski's covariance model [23, 24], regions of potentiation and depression are
Search WWH ::
Custom Search