Biomedical Engineering Reference
In-Depth Information
w
ic
t
ð
þ
1
Þ ¼
w
ic
ðÞ þ
t
a
i
;
;
(4.22)
w
ij
t
ð
þ
1
Þ¼
w
ij
t
ðÞ
a
i
;
if
w
ij
t
ð
þ
1
Þ <
0
w
ij
t
ð
þ
1
Þ ¼
0
;
where
w
ij
(
t
) is the weight of the connection between the
i
-neuron of the
A
-layer and
the
j
-neuron of the
R
-layer before reinforcement,
w
ij
(
t
+ 1) is the weight after
reinforcement, and
a
i
is the output signal (0 or 1) of the
i
-neuron of the
A
-layer.
Taking into account the small number of active neurons, it is convenient to
represent the binary vector
A
not explicitly but as a list of numbers of active
neurons. For example, let the vector
A
be:
A
¼
00010000100000010000
:
(4.23)
The corresponding list of the numbers of active neurons will be 4, 9, and 16. This
list is used to save the image codes in compact form and to calculate quickly the
activity of the neurons of the output layer. Thus, after executing the coding
procedure, every image has a corresponding list of numbers of active neurons.
The training process is carried out iteratively. After representation of all the
images from the training subset, the total number of training errors is calculated. If
this number is higher than 1% of the total number of images, then the next training
cycle is continued. If the error number is less than 1%, the training process is
stopped. The process is also stopped when the cycle number is more than a initially
prescribed value. In our experiments, we used 30 cycles with the MNIST database
and 200 cycles with the ORL database. The flowchart of the algorithm is shown in
Fig.
4.11
(Parts 1 and 2).
It is known [
3
] that the performance of the recognition systems could be
improved due to the distortions of the input image during the training process. In
[
8
], we showed that further improvement could be obtained if the distortions are
used not only in the training process, but in the recognition process, too.
In the MNIST database, we used only skewing as image distortion. In our
experiments, we used three skewings (11.3
, 18.5
,26.5
) to the left side and
three skewings of the same angles to the right side. Thus, the total number of
distortions in the training process was six. In the recognition process, we tested
three variants of the system: without distortions, with two distortions, and with four
distortions. In the ORL database, we used the shifted images for training and did not
use distortions for recognition.
4.7 Results Obtained on the MNIST Database
The MNIST database contains 60,000 samples in the training set and 10,000
samples in the test set. The results of the experiments are presented in Table
4.2
.
The values presented in this table correspond to the number of errors in the
recognition of 10,000 test samples. We conducted five experiments with different
sizes of the rectangle of dimensions
h
*
w
(7 * 7, 9 * 9, 11 * 11, and 13 * 13).
