Biomedical Engineering Reference
In-Depth Information
Hence, if we can nd another set of standardized vectors, which retain
the inner products and are the orthogonal transforms of
i
's and s in the
previous example, the same results can be predicted for LARS.
The standardization can be incorporated according to the following. The
main idea is that an n-dimensional linear space can be treated as a subspace
of R
n+1
, which is orthogonal to vector (1; 1; :::; 1). Letfb
0
; b
1
; :::; b
n
gdenote
an orthonormal basis of R
n+1
, with b
0
=
n+1
(1; 1; :::; 1)
T
. Denote the unit-
norm vectors s = (s
1
; s
2
; :::; s
n
)
T
and
i
= (
i1
;
i2
; :::;
in
)
T
, i = 1; 2; :::; m.
Dene s
1
p
P
n
j=1
s
j
b
j
,
P
n
j=1
ij
b
j
, i = 1; 2; :::; m. One can easily
0
=
0
i
=
0
0
i
0
0
j
verify thaths
;
i=hs;
i
ifor 1im, andh
i
;
i=h
i
;
j
ifor
1i; jm. Hence, applying LARS to s
0
and
0
i
's will produce the same
result as in the rst case study. It is not hard to verify that s
0
and
0
i
's are
standardized. Therefore, the conclusions in our case study can be extended
to the case with standardized response and covariates.
Theorem 5: There exists an orthogonal transform that can be applied to
the previous example to create a case in which all the covariates and the
response are standardized, and LARS select all the covariates outside the
optimal subset before it chooses any covariate inside the optimal subset.
4.1.2. To Create a Dramatic Presentation
The foregoing example is developed in a fairly general form, without speci-
fying the controlling parameters: A and m. To see how dramatic an example
can be, let us consider the case where A = 10 and m = 1; 000; 000. Based on
the previous analysis, LARS will select the rst 999; 990 covariates before it
selects any of the last ten covariates. At the same time, the optimal subset
is formed by the last ten covariates.
4.2. Variable Selection with Orthogonal Model Matrix
In order to gain more insights, a case in which is orthogonal is considered.
This example has been studied in the original LARS paper
14
. The purpose
of restating it here is to illustrate that there is a case in which LARS nd
the type-I optimal subset. I.e., (P0) and (P1) coincide.
Theorem 6: Let
x
1
denote the solutions to (P0) and (P1), re-
spectively. When is orthogonal, we have
e
x
0
and
e
j
p
0;
if jz
i
0
;
p
e
x
0;i
=
z
i
;
if jz
i
j>
0
;
Search WWH ::
Custom Search