All Relevant Feature Selection Methods and Applications - Feature Selection for Data and Pattern Recognition - page 11

Fig. 2.1 The illustration for the toy problem. The upper panel shows projection of the system on the
plane

( X 1 X 2 )

.The lower panel shows the importance of variables in random forest classifier. Solid
squares correspond to variables used to generate the decision variable, solid circles correspond
to combinations of X 1 and X 2 and open circles correspond to random variables. It is clear that
importance of variables obtained from random forest can be used to discern informative and non-
informative features. Nevertheless, the importance ranking does not allow to detect the variables
used for generation of the decision variable

a value of the decision variable. The first task can be tackled using the all-relevant
feature selection approach. The solution of the second task, which generally is much