Modeling with Neural Networks: Principles and Model Design Methodology - Neural Networks: Methodology and Applications - page 190

2.10.3.3 Fisher Distribution

If Y 1 is a Pearson variable with N 1 degrees of freedom, and if Y 2 is a
Pearson variable with N 2 degrees of freedom, the random variable Z =
( Y 1 /N 1 ) / ( Y 2 /N 2 ) has a Fisher distribution with N 1 and N 2 degrees of free-
dom.

2.10.4 Input Selection: Fisher's Test; Computation of the
Cumulative Distribution Function of the Rank of the Probe
Feature

2.10.4.1 Fisher's Test

We first describe the use of Fisher's test for model selection.
We assume that the measurements of the quantity of interest can be mod-
eled as the realizations of a random variable such that Y p = ζ T w p + Ω ,where
ζ is the vector of the variables of the model (of unknown dimension), where
w p is the vector (nonrandom but unknown) of the parameters of the model,
and where Ω is an unknown random Gaussian vector, with zero mean. Thus,
one has

E ( Y p )= ζ T w p .
We want to find a model g , from a set of N measurements

y p , k =1to
N} ; y p is the N -dimension vector whose components are the y p . The model
depends on the training set: therefore, it is also a realization of a random
variable G .
Assume that a set of Q variables, which contains certainly the measurable
variables that are relevant to the modeling of the quantity of interest, has
been found. A model that contains all relevant variables is called a complete
model. Then a model is sought, of the form

{

G Q = ζ T W Q ,

Q

where ζ Q is the input vector of the model (of dimension Q +1 since, in addition
to the relevant variables, a component equal to 1 is present in the input
vector), and where W is a random vector, which depends on the realization
of the vector Y p that is used for the design of the model. That model is said to
be true: there exists certainly a realization w p of the random vector W such
that g Q = E ( Y p ).
In the present chapter, the vector of the parameters of the model was
always found by minimizing the least squares cost function (except when using
weight decay) J ( w )= k =1 ( y p −

2 , where w
is a realization of the vector of parameters W , ζ k is the vector of the Q +1
inputs for example k ,andwhere g Q ( ζ ,w ) is the vector of the realizations of
G Q for the N measurements.

g Q ( ζ k , w )) 2 = y p − g Q ( ζ , w )