Biomedical Engineering Reference
In-Depth Information
esting fact is that a majority of these criteria can be unied under (P0),
wherekyxk
2
is the residual sum of squares (denoted by RSS(x)) un-
der the coecient vector x, and constant
0
depends on the criteria. The
following summarizes some well-known results:
Paper [1] denes its criterion by maximizing the expected log-
likelihood E
X;
^
(log f(Xj
^
)), where
^
is the estimation of param-
eter , f(Xj) is the density function. This is equivalent with max-
imizing the expected Kullback-Leibler's mean information for the
discrimination between f(Xj
^
) and f(Xj), i.e.: E
X;
^
(log
f (Xj
^
)
f (Xj)
),
for a known true . Under a Gaussian assumption in the linear re-
gression, the above leads to the Akaike information criterion (AIC)
that minimizes
AIC =
RSS(x)
2
+ 2kxk
0
;
where
2
is the noise variance, and other notations have been de-
ned at the beginning of this section. It is a special case of (P0)
by assigning
0
= 2
2
.
Mallows' C
p
(see [35, 24]), which is derived from the unbiased risk
estimation, minimizes
1
^
C
p
=
2
RSS(x) + 2kxk
n;
0
2
=
2
is assumed,
Mallows' C
p
is equivalent to AIC. Again, C
p
is a special case of
(P0).
Motivated by the asymptotic behavior of Bayes estimators,
Bayesian information criterion (BIC)
43
chooses to select the model
that maximizes
where ^
is an estimate of parameter . When ^
log f(Xj
^
)
1
2
log nkxk
0
:
Here, under the squared error loss and the Gaussian model assump-
tion with known variance
2
, BIC is to minimize
RSS(x)
2
BIC =
+ log nkxk
0
:
The above is again a special case of (P0) by assigning
0
=
2
log n.
The equivalence between BIC and the minimum description length
(MDL) is well known, see Hastie, et al.
27
. Hence, MDL is a special
case of (P0) as well.
Search WWH ::
Custom Search