Information Technology Reference
In-Depth Information
LSTM memory block
LSTM memory cell
LSTM memory cell
Output
gate
Input
gate
Fig. 6.15
An LSTM memory block with two memory cells and two gates.
LSTM memory cell
g
CEC
h
Input gate
Output gate
Fig. 6.16
An LSTM memory cell (also visible are the input and output gates).
6.2.2.2
MEE for LSTM
Let the error E k = T k
Y k represent the difference between the target T of the
k output neuron and its output Y ,atagiventime τ (not explicitly given to
keep the notation simple). The MMSE of the variable E k will be replaced by
its MEE counterpart. First it is necessary to estimate the PDF of the error.
For this purpose, the Parzen window approach is used (see Appendix E)
allowing to write the error PDF estimate as:
K e k
,
n
1
nh
e i
f ( e k )=
(6.23)
h
i =1
where, as in expression (3.2), h represents the bandwidth of the kernel K and
n is the number of neurons in the output layer.
Using (6.23) one then applies the
H R 2 estimate (F.9) to the MEE approach
log 1
n 2 h
.
K e ( i )
n
n
e ( u )
H R 2 ( e ( j )) =
(6.24)
h
i =1
u =1
Search WWH ::




Custom Search