Information Technology Reference
In-Depth Information
the authors, the most ecient model (as a function of learning time and per-
formance) is the approach that connects metaplasticity and Shannon informa-
tion theory, which establishes that less frequent patterns carry more information
than frequent patterns [14]. This model then defines artificial metaplasticity as
a learning procedure that produces greater modifications in the synaptic weights
with less frequent patterns than frequent patterns, as a way of extracting more
information from the former than from the latter. As Biological metaplasticity,
AMP then favors synaptic strengthening for low-level synaptic activity, while
the opposite occurs for high level activity. The model is applicable to general
ANNs. Andina
. propose general AMP concepts for ANNs, and demonstrate
them over Radar detection data [1].
et al
4 Backpropagation Algorithm and AMP
The AMP implementation applied tries to improve results in learning conver-
gence and performance by capturing information associated with significant rare
events.
It is based on the idea of modifying the ANN learning procedure such that
un-frequent patterns which can contribute heavily to the performance, are con-
sidered with greater relevance during learning without changing the convergence
of the error minimization algorithm
. It is has been proposed on the hypothe-
sis that biological metaplasticity property maybe significantly due to an adap-
tation of nature to extract more information from un-frequent patterns (low
synaptic activity) that, according to Shannon's Theorem, implicitly carry more
information.
4.1 Mathematical Definitions
Let us define an input vector for a MLP with
n
inputs (bias inputs are assumed
to exist and be of fixed value set to 1):
x ∈ R
n
,where
R
n
is the n-dimensional
space, i.e.
x
R
1
,
i
=(
x
1
,x
2
, ...., x
n
)
,
x
i
∈
=1
,
2
, ..., n
; and its corresponding
j
outputs given by vector
y
=(
y
1
,y
2
, ..., y
n
)
,
y
i
∈
(0
,
1)
,
j
=1
,
2
, ..., m
[15]. Let
us consider now the random variable of input vectors
X
=(
X
1
,X
2
, ..., X
n
)
with
probability density function (pdf)
f
X
(
. The strategy of
MLP learning is to minimize an expected error,
E
M
, defined by the following
expression:
x
)=
f
X
(
x
1
,x
1
, ..., x
n
)
E
M
=
ε
{
E
(
x
)
}
(1)
where
E
is the expression of an error function between the real and the
desired network output, being respectively
Y
(
x
)
=
F
(
X
)
,withpdf
f
Y
(
y
)
and
Y
d
the desired output vector, and
F
is the nonlinear function performed by the
MLP. The symbol
ε
represents the mathematical expectation value, that is,
(
X
)
E
M
=
E
(
x
)
f
X
(
x
)
dx
(2)
R
n
Search WWH ::
Custom Search