Information Technology Reference
In-Depth Information
fu
we
w
j
'
K
x ef
K
c
ux
KG
x
j
i
j
j
i
j
i
w
u
j
where
G can be expressed as
w
S
j
G
ef u
c
w
.
(3.8)
j
j
j
u
j
The derivation
fu
c
of the selected activation function (3.1) is
j
w
fu
j
ª
º
ª
2
º
fu
c
J
1tanh
2
J
u
J
1
y
,
(3.9)
¬
¼ ¬
¼
j
j
j
w
u
j
and the corresponding weight updates (3.7)
2
' ,
we
KJ
1
y
x
(3.10)
i
j
j
i
with
KJ! .
Note that the weight update stabilizes if
0
y
approaches -1 or +1, since the
ww
, equal to
2
partial derivative
j
y
and its minima for r . However, if the sigmoidal activation function is used and if
it is unipolar, described by
yu
J
1
y
, reaches its maximum for
0
j
j
j
1
yf
u
,
(3.11)
j
j
1exp
J
j
then
w
fu
j
c
fu
J
y
1
y
.
(3.12)
j
j
j
w
u
j
Therefore, the weight increment takes the form
' .
we
KJ
y
1
y
x
(3.13)
i
j
j
j
i
Search WWH ::
Custom Search