Cross-Validation, the Jackknife, and the Bootstrap: Excess Error Estimation in Forward Logistic Regression - Common Errors in Statistics - page 179

0 10 20 30 40 50

-0.052
-0.045
-0.039
-0.032
-0.028
-0.019
-0.013
-0.006
0.000
0.008
0.013
0.019
0.026
0.032
0.039
0.045
0.052
0.058
0.064
0.071
0.077
0.084
0.090
0.097
0.103
0.110
0.116
0.129

*
***
* ** **
****
********
**********
*****************
**********************
***************************

*

**********************************

*******************************************
********************************************

**************************************

*******************************
**********************

**********************
****************
***********
****
* **
*

** ******* ***** ****** *******

* *

FIGURE 1 Histogram of Bootstrap Replications for Gregory's Rule. The his-
togram summarizes the 400 bootstrap replications of R* that are used in estimat-
ing the expected excess error of Gregory's rule for predicting death in chronic
hepatitis. Values of R* range from -0.045 to 0.116, with mean 0.039, standard
deviation 0.027, and quantiles R (.05) =-0.006 and R (.05) = 0.084.

(

*

) ==

ˆ ˆ

ˆ ˆ ,

ERFFr , r

ˆ *

~ ˆ

boot •

FF

and variance, say, s 2 . Figure 1 shows that this population is close to
normal, so

ˆ ˆ

12

r -£

•

r 2 s 400 ,

400

with high probability. Approximating s 2 with

400

1
400

* -

2

[ ] = (

2

 Rr

)

ˆ ˆ

s 400

2

= 0 027 .

b 400

- 1

b - 1

gives

-£ (

)

ˆ ˆ

12

r r 2 0 027 . 400 = 0 0027 . ;

400 •

r r r

so with high probability, 400 is within 0.0027 of • =

boot .