Biomedical Engineering Reference
In-Depth Information
ing example with standardized covariates and signal is presented later in
Section 4.1.1. Section 4.1.2 shows how to use the result to come up with a
dramatic example for presentation.
The rst example is generated as follows. Let
i
R
n
; i = 1; 2; :::; m,
denote the ith column of the model matrix . Hence, = [
1
;
2
; :::;
m
].
Let
i
2
R
n
; i = 1; 2; :::; m, denote the dirac vector taking 1 at the ith posi-
tion and 0 elsewhere. For i = mA + 1; mA + 2; :::; m, let
i
=
i
, where
A is a positive integer. Consider a special signal s =
2
P
m
i=mA+1
i
.
Obviously, in this case, the optimal subset isfmA + 1; :::; mg. For the
rst mA columns of , make
j
= a
j
1
p
A
j
, where 1jmA
and a
j
+ b
j
= 1. Note
i
's and s are all unit-norm vectors. From now on,
for simplicity, we always assume 1jmA and mA + 1im.
It is easy to verify that
s + b
j
p
hs;
j
i= a
j
and hs;
i
i= 1=
A:
p
In this example, we choose 1 > a
1
> a
2
>> a
mA
> 1=
A > 0:
Now we consider the procedure of LARS. In the rst step, since
1
has
the largest inner product with s, evidently column
1
will be chosen. The
residual will be r
1
= sc
1
1
, where c
1
is the coecient to be determined.
The following result about the consequent step in LARS is proved in the
Appendix of [28].
Lemma 2: In the consequent step of LARS, covariate
2
is chosen, with
c
1
=
a
1
a
2
1a
1
a
2
:
Hence, the residual of the rst step becomes
r
1
= sc
1
1
= s
a
1
a
2
1a
1
a
2
(a
1
s + b
1
1
)
b
1
1a
1
a
2
s
(a
1
a
2
)b
1
1a
1
a
2
=
1
b
1
1a
1
a
2
[s
a
1
a
2
b
1
=
1
]:
Note that in LARS, only the direction of a residual vector determines the
selection of the next covariate(s). The amplitude of a residual vector does
not change the variable selection. Hence, we introduce a surrogate residual
with a simpler form:
r
1
= s
a
1
a
2
b
1
e
1
:
Search WWH ::
Custom Search