SOME RECENT RESULTS IN MODEL SELECTION - Quantitative Medical Data Analysis Using Mathematical Tools and Statistical Techniques - page 31

At the same time, (P1) has been proposed in statistics as a way of sub-
set selection. The method is named Lasso 47 . An interesting recent develop-
ment | the least angle regressions (LARS) 14 | demonstrates that certain
greedy algorithms can reveal the solutions to (P1) with varying values of
1 , based on the idea of homotopy 39 . More recent analysis demonstrates
further that greedy algorithms can literally render the entire solution path
in a large class of problems, referring to Hastie, et al. 26 and the references
therein. A recent conference presentation 32 gives the most succinct solution
in generating solution paths, utilizing a homotopy continuation method 40
and an analysis of subdierential. A standard reference for the background
of this material is Rockafellar 42 .
4. Case Study
To illustrate further the necessity and feasibility of deriving equivalence
conditions between (P0) and (P1), we describe two extreme examples. In
the rst example, solutions of (P1) and (P0) completely disagree. In the
second example, (P1) and (P0) share the same subset.
4.1. An Extreme Example for the Least Angle Regressions
Least Angle Regression 14 is a forward variable selection method. An ex-
tensive manual regarding forward selection can be found in Atkinson, et
al. 2 . As been indicated previously, LARS can give the solution path of (P1).
However, this homotopy does not guarantee that LARS always reveal the
optimal solutions of (P0); i.e., (P0) and (P1) could disagree. In this sub-
section, we present one particular case, in which LARS choose wrongly in
the rst iteration and end up correcting it ineciently. As a result, LARS
do not include the correct covariates until the last step. Initially, such an
example motivated us to consider the conditions of equivalence.
Details of LARS algorithm can be found in Efron, et al. 14 . In a nut-
shell, LARS start with zero coecients, select the most correlated covariates
with the signal (i.e., the response) s, then move along the direction that is
equiangular among the selected covariates until some other covariates have
as much correlation with the current residual, add these new covariates un-
der consideration and move along the new equiangular direction. When the
covariates and the response are standardized to have mean 0 and unit norm,
correlation between vectors is proportional to the inner product. In the fol-
lowing, for clarity, we rst give an example with nonstandardized vectors,
and choose the covariates according to the inner products. The correspond-