Databases Reference
In-Depth Information
the learning algorithm. Depending on the evaluation criteria three different
approaches for the feature selection methods were reported in Ref. 37:
•
wrapper approach
•
filter approach
•
embedded approach.
Wrapper Approach:
In this approach, the selection algorithm searches
for a good subset of features using some induction algorithm. Once the
induction algorithm is fixed, train with these feature subset by the search
algorithm and estimate the error rate. The error rate can be assigned as the
value of the evaluation function of the feature subset. Thus in this approach
the selection of feature is based on the accuracy of the classifier.
Kohavi and John
38
introduced wrappers for feature selection and the
approach is tailored to a particular learning algorithm and a particular
training set. The selection algorithm in a wrapper approach depends on
both the number of features and number of instances.
The major drawback of the wrapper approach would be feeding with an
arbitrary feature into the classifier may lead to biased results and therefore
the accuracy can not be guaranteed. Another drawback is that for a large
set of features trying all possible combinations to feed the classifier may
not be feasible. Therefore, some researchers are motivated to alleviate the
excessive loading of the training phase avoiding the evaluation of many
subsets exploiting intrinsic properties of the learning algorithms.
39
Filter Approach:
This approach evaluates the goodness of the feature
set in regards only to the intrinsic properties of the data, ignoring the
induction algorithm. Since filter is applied to the algorithm to select relevant
features considering the data and the target concept to be learned, the
approach is referred as filter approach. Obviously, filter method would be
faster than the wrapper approach. Filter method feature selections are
appropriate for the huge database while wrapper methods are infeasible
40
Embedded Approach:
This approach has been identified in.
10
In this
case the feature selection process is done inside the induction algorithm
itself.
5.3.4.
Particle swarm optimization
Particle swarm optimization technique is considered as one of the modern
heuristic algorithm for optimization introduced by James Kennedy and
Eberhart in 1995.
41
A swarm consists of a set of particles moving around
