Modeling Methodology: Dimension Reduction and Resampling Methods - Neural Networks: Methodology and Applications - page 226

We denote by j the rank of the bootstrapped base and by i the iteration
on the number of cycles; the average quadratic learning and testing errors are
represented by the following two tables:
⎡

⎤ ⎡ ⎤

ε ∗ 1
1

ε ∗ 2
1

ε ∗B
1

ε 1 ε 1 ε 1 ··· ···

⎣ ⎦ ⎣ ⎦

ε ∗ 1 2 ε ∗ 2 ε ∗B

2

ε 2 ε 2 ε 2 ··· ···

2

.

. . . . . .

. . . . . .

ε ∗ 1
N c

ε ∗ 2
N c

ε ∗ N c ε 1

N c

ε 2
N c

ε N c ··· ···

learning error testing error

After that phase, NeMo determines the number of cycles by application of
heuristics based on game theory. A first pessimistic player considers the worst
possible situation on the test error, for each number of cycles value,

=Max b ε i .

ε Max
i

The second player then determines the number of cycles that corresponding
to the worst situation obtained; that is, the number of cycles that corresponds
to the maximum testing error,

=Arg i Min ε Max

.

N optimal
c i

That strategy for the selection of N optima c may be relaxed by adopting
only a fraction of the set of B training cycles. To make it more robust, just
exclude the outliers, that is, training situations that differ greatly from the
average. By default, NeMo determines the optimum number of cycles on the
90th percentile of the test error.

Percentile

The α th percentile corresponds to the interval made up of the values for which
the distribution function is smaller than α : a fraction (1 − α ) of the maximum

values is excluded.
The optimum number of cycles may also be estimated by the tri-median
method, which is more stable but more risky since 25% of cases are rejected:
the last quartile that corresponds to the largest test errors.

Quartile

If F is the distribution function, the 1st and 3rd quartile Q 1 and Q 3 and the
median Q 2 are defined respectively by F (Q 1 )=0 . 25, F (Q 2 )=0 . 5, F (Q 3 )=
0 . 75.