Information Technology Reference
In-Depth Information
4R su s
4.1
Evaluation of the Adaptive Learning Rate
To evaluate the improvements caused by introduction of the adaptive learning rate as
described in Sec. 2.1, an RNNPB was trained 1000 times with two 1-D sequences (Eq.
9 and 10). The results are statistically evaluated using a t-test. To compensate for the
sample size bias, the optimal sample size was determined based on the mean value and
the standard deviation of the data. This optimal sample size was used to draw 10,000
random subsets of the data, which were subsequently evaluated to obtain an average
p-value for the t-test. The results are summarized in Tab. 1. The modifications lead, on
average, to a 22-fold speedup of the training times ( t (5) =
17 . 13 , p =0 . 000 ) for this
particular example. Also the number of recognition steps has improved significantly
( t (20) =
3 . 55 , p =0 . 002 ). However, no significant changes of the retrieval accuracy
measured with the mean squared error (MSE) can be found.
Ta b l e 1 . Statistical evaluation of the adaptive learning rate. Mean values and standard deviations
are shown, significant changes (t-test, p< 0 . 005 ) are marked bold.
Modified
Classical
Factor
RNNPB
RNNPB
5,520 ( ± 1,713)
±
Total steps
122,709 (
20,027)
22.2
Total time
34 s ( ± 10)
751 s ( ± 124)
22
4.3 × 10 4
( ± 1.2 × 10 3 )
5.5 × 10 4
( ± 3 × 10 4 )
MSE sin
-
4.7 × 10 4
( ± 8.7 × 10 4 )
× 10 4
( ± 1.9 × 10 4 )
MSE sinc
.
-
Recognition
192 ( ± 85)
284 ( ± 101)
1.48
steps
Plotting the average MSE against the number of steps needed until the convergence
criterion is reached, further highlights the drastic improvement in speed (Fig. 5). The
error, shown separately for both sequences, decreases for both algorithms in a simi-
lar manner. However, the adaptive version looks 'compressed' in comparison to the
classical algorithm. In addition, the fluctuations are reduced, indicating a more stable
behavior of the modified RNNPB.
4.2
Classification Using All Object Categories for Training
In the first experiment the modified recurrent neural network with parametric bias was
trained with the bi-modal prototype time series of all eight object categories (Sec. 3.3).
During training, the PB values for the respective categories emerged in an unsupervised
way. This means, the two-dimensional PB space self-organizes based on the inherent
properties of the sensory data that was presented to the network. Hence, objects with
similar dynamic sensory properties are clustered together. This can be seen in Fig. 6. For
instance, the learned PB vectors representing star- and circular-shaped objects, either
light-weight (white symbol) or heavy (black symbol), are located in close proximity,
 
Search WWH ::




Custom Search