Modeling Methodology: Dimension Reduction and Resampling Methods - Neural Networks: Methodology and Applications - page 223

The confidence interval may be also calculated by the bootstrap, with the
following algorithm.
Using the initial sample, we simulate new samples, known as “boot-
strapped” bases, the size of which is n , by random selection with replacement.
For instance, consider the initial sample defined above x =

{ 16 , 12 , 14 , 6 , 43 , 7 ,

0 , 54 , 25 , 13 . By random selection with replacement, we obtain for exam-

ple the following bootstrapped base x ∗ =

}

,in
which some values of the initial sample are missing, whilst others appear sev-
eral times. Several samples are thus simulated. For each sample, an average is
computed. The confidence interval at 95% is defined for that set of averages.
The simulation produces the following:

{ 54 , 0 , 16 , 7 , 43 , 54 , 0 , 25 , 25 , 6 }

9 <µ< 26 .

It should be noted that the interval obtained using the bootstrap is virtually
identical to the 95% confidence interval computed above from the central limit
theorem.

Bootstrap—General

The bootstrap does not require any assumption on the underlying statistical
distribution.
The bootstrap may therefore be applied to all estimators other than the
average, such as the median, the coe cient of correlation between two ran-
dom variables, or the largest eigenvalue of a variance-covariance matrix, for
example. For those estimators, no analytical expression is available for the
standard error or the confidence interval. The only applicable methods are
the so-called resampling methods, which consist in the simulation of samples
such as the bootstrap or the jackknife [Efron 1993].

3.6.2 Bootstrap Estimation of the Standard Deviation

Consider a random variable X that obeys the probability distribution F .
We want to estimate a parameter θ of F . θ is estimated from an n -sample
x = .Wedenoteby F the empirical distribution, and by

θ = s ( x ) the estimation of θ from sample x . The algorithm is as follows:

{ x 1 ,x 2 ,...,x n }

1. Select B bootstrapped n -samples, x ∗ 1 , x ∗ 2 ,..., x ∗B ,eachofthembeing
obtained from the initial sample x by n random selections with replace-
ment
2. For each bootstrapped n -sample, compute a replica of the estimate of θ
as

θ ( b )= s ( x ∗b ) , b =1 , 2 ,B.