Weighting of Features by Sequential Selection - Feature Selection for Data and Pattern Recognition - page 79

Fig. 5.2 DRSA classification accuracy observed in sequential forward selection process, in relation
to the number of considered features, for both minimal cover (MCDA) and all rules on examples
(FDA) decision algorithms
This approach is also the one to be used when the number of possible features is
practically infinitive—as the search cannot be endless, we need to limit the number
of subsets to be tested somehow and from them select the best.
Once the search path of forward sequential selection is completed the goal of
feature weighting is achieved by observing the order in which attributes were chosen,
as listed in the (b) column of Table 5.1 , with the frequency of occurrence of “that”
at the top of the list and “;” at the bottom.
In the presented research to the central search for important attributes one more
element was introduced, which added one more dimension to the search space at
each stage, and it was a preference order for an attribute. In the stylometric domain
preference orders should be understood as associating certain, higher or lower, fre-
quencies of usage of linguistic elements with specific authors. Although undeniably
such relationships exist and enable authorship attribution in the first place, a priori
knowledge about them, ready and applicable to any writers, any texts, any samples,
does not exist. In its absence preference orders can be arbitrarily assumed, or they can
be adjusted through some processing [ 40 ]. But when we actively search for subsets of
relevant attributes it is more natural to extend this search to include not only selection
of variables but their preference orders as well. As a result the obtained solutions
are closer tailored to specifics of the classification task, which is visible in higher
predictive accuracy when we compare them to the case of rule classifiers induced
for attributes with arbitrarily assigned preference orders, tested within sequential
backward elimination procedures described in the next section.