Parameter Effects in the Game of Chung Toi - Design of Experiments for Reinforcement Learning - page 160

Table A.3 Experiment results.
Experiment Max % wins Game # at

max % wins

ʱ Emb 92.6 5000

0 . 90 91.8 10,000

ʳ 0 . 75 91.8 4000

games 100 k 91.2 80,000

base 90.4 6000

batch 1 /mom 90.2 10,000

tanh 90.2 10,000

ʱ anneal 89.0 2000

ʱ anneal , Emb 88.8 3000

batch 10 /mom 88.2 6000

tanh LeCun 86.4 1000

w LeCun 84.2 8000

nodes 20 83.2 2000

anneal 83.0 9000

ʻ 0 . 4 76.2 3000

nodes 80 74.0 4000

ʻ 0 . 9 71.4 1000

￿ Hidden nodes : The number of nodes in the hidden layer seemed to have a pro-

found effect on the learning ability of the network (Fig. A.3 a). In both the nodes 20
and nodes 80 experiments, the performance was poor overall. The small network
with only 20 hidden nodes was likely unable to approximate the true state-value
function, and the network with 80 hidden nodes may require additional training
beyond the 10,000 games used in this comparison.

￿ Weight initialization : The experiment using the weight initialization ( w LeCun )

method described in LeCun et al. (1998) had lower performance, compared to
base , at every evaluation point during training, however, the performance was
steadily increasing throughout (Fig. A.3 b).

￿ Hidden transfer function : The experiments comparing the different hidden trans-

fer functions performed worse throughout most of the training compared to the
base experiment (Fig. A.3 c), although performance did increase relatively quickly
and was somewhat stable. Despite this, the tanh transfer function was able to
achieve a maximal performance of 90.2 % at the end of training.