Biomedical Engineering Reference
In-Depth Information
Penalties
5
4.5
4
3.5
3
2.5
2
1.5
SCAD
L
1
L
2
1
0.5
0
−5
−4
−3
−2
−1
0
1
2
3
4
5
q
Fig. 1.
Penalty Functions
8
<
:
jj;
if 0jj< ;
(a
2
1)
2
(jja)
2
2(a1)
;
if jj< a;
p
(jj) =
(a+1)
2
2
;
ifjja:
The SCAD uses two tuning parameters, and a. Fan and Li
15
suggested
xing a = 3:7 based on a Bayesian argument. They also found that in terms
of empirical performance in simulations, the SCAD estimate using a = 3:7
was as good as the SCAD estimate with the value of a chosen by GCV.
The SCAD penalty, the L
1
and L
2
penalty functions are depicted in Fig-
ure 1. The SCAD estimator is similar to the LASSO estimator since it gives
a sparse and continuous solution, but the SCAD estimator has lower bias
than LASSO. Directly applying Theorems 1 and 2 of Fan and Li
15
, it can
be shown that under certain regularity conditions and with proper choice
of penalty functions and tuning parameter, the SCAD-penalized estimate
is
p
nconsistent and possesses the oracle property asymptotically. In par-
ticular, when !0 and
p
n!1, maximizing the penalized likelihood
Search WWH ::
Custom Search