Weighting of Features by Sequential Selection - Feature Selection for Data and Pattern Recognition - page 68

Chapter 5
Weighting of Features by Sequential Selection

Urszula Sta nczyk

Abstract Constructing a set with characteristic features for supervised classification
is a task which can be considered as preliminary for the intended purpose, just a step
to take on the way, yet with its significance and bearing on the outcome, the level
of difficulty and computational costs involved, the problem has evolved in time to
constitute by itself a field of intense study. We can use statistics, available expert
domain knowledge, specialised procedures, analyse the set of all accessible features
and reduce them backward, we can examine them one by one and select them for-
ward. The process of sequential selection can be conditioned by the performance
of a classification system, while exploiting a wrapper model, and the observations
with respect to selected variables can result in assignment of weights and ranking.
The chapter illustrates weighting of features with the procedures of sequential back-
ward and forward selection for rule and connectionist classifiers employed in the
stylometric task of authorship attribution.

· · ·

Keywords Weighting Ranking of features Sequential selection Forward selec-

· · · · ·

tion Backward selection DRSA ANN Stylometry Authorship attribution

5.1 Introduction

In order to arrive at a set of characteristic features which are relevant for a task and
give some satisfactory predictive accuracy for a classification system employed in
supervised pattern recognition [ 20 ], we can either start with the empty set and then
in the process of forward selection add some number of attributes to it, or we can
execute backward elimination of features from some original set, chosen basing on
expert domain knowledge, some other algorithms or measures, or even randomly.
We can also attempt to go in both directions at the same time, mixing elimination
with selection of features [ 19 ].