SOME RECENT RESULTS IN MODEL SELECTION - Quantitative Medical Data Analysis Using Mathematical Tools and Statistical Techniques - page 37

parameter . Value g 0 ( 0 )=2 is called the generalized degrees of freedom in
Ye 52 .
The above two are merely the representative examples of many inter-
esting approaches.

5.3. Back Elimination

Subset selections include at least three basic approaches: forward selection,
backward elimination, and all subset selection. Problem (P0) is an all sub-
set selection method. The greedy algorithms that have been discussed in
this chapter are assumed to be forward selection algorithms.
In Couvreur and Bresler 8 , a very interesting result is proved for the
backward elimination. It is shown that under certain conditions, back elim-
ination nds the solution of (P0). Such a result reveals the properties of
problem (P0) from another angle.

5.4. Other Greedy Algorithms and Absolutely Optimal
Subset in Variable Selection

We have treated LARS as a stepwise algorithm. Other greedy algorithms
have made signicant impact in other elds (e.g., signal processing). Two
representative ones are matching pursuit (MP) 9;34 and its improved version
{ orthogonal matching pursuit (OMP) 41 . MP and OMP do not generate
the regularized solution path, while a version of LARS does. However, the
intensive research eort following MP and OMP will provide researchers
powerful tools.
Researchers have studied on the subsets that are unconditionally con-
current optimal, i.e., its concurrent optimality depends on neither the co-
ecients nor the corresponding residuals. The representative works include
[10, 49], and [48]. The concept of exact recovery coecient (ERC) 48 has

inspired many recent works.
Note that in our sucient conditions, both coecient and residuals are
taken into account. This is due to the dierent emphasis of the problems.
Compared with our works, the results mentioned in the last paragraph can
be considered as an analysis of the worst cases.

5.5. Model Selection versus Variable Selection

We may think that model selection and variable selection are interchange-
able. It is pointed out in [30] that there are dierences between the two