Information Technology Reference
In-Depth Information
objectives at the same time [14,4]. Nevertheless, most experiments fall into evolution-
ary algorithms test sets and they are not based in popular MOEA implementations such
as NSGA-II [6] and SPEA2 [33].
The Hold-Out validation is a suitable approach to evaluate the case selection methods
due to large size of initial data base. Experiments as [22,21,25,23,7,16] uses Hold-Out
repeatedly a determined number of times. Furthermore, some studies try to analyse
the effects of case memory size in the classifier accuracy. However, this studies are
restricted to K-NN based classifiers. In this cases, the case selection process finishes
when a concrete (and pre-established) size of the case memory is reached [21,25,16].
Although Hold-Out is appropriate for large cases sets, it could present a high variance.
Therefore, some researchers prefer Cross-Validation [31].
3
Evaluation Methodology and Case Selection Methods
3.1 Notation
Let
C
be the universe of all possible cases in a give domain and
c
∈
C
be a particular
case. We assume that a case is a vector of
n
attributes
x
=(
x
1
,
···
,x
n
)
, where each
attribute could describe a quantitative or qualitative value.
Let also define
M =
℘
(
C
)
as the space of possible case memories and
M
∈
M
a
particular case memory. Therefore, a case
c
of
M
is denoted by
c
∈
M
.
be a partition of
M
(
i
=1
Let
{
M
i
,i
∈{
1
,
···
,f
}}
M
i
=
M
,
∀
i, jM
i
∩
M
j
=
∅
).
We define the complement of
M
i
(element of a partition of
M
)as
M
i
=
M
M
i
.
A case selection method can be defined by the function
σ
as follows:
σ
: M
→
M
(1)
. For the sake of clarity, we denote
M
σ
as
σ
where
. In Section 2,
the implementation of
σ
functions proposed in the literature are briefly reviewed and
Section 3.3 describes in depth some of them.
Given
M
σ
we define the reduction rate as:
|
σ
(
M
)
|≤|
M
|
(
M
)
: M
σ
→{
0
ρ
,
1
}
(2)
M
σ
|
M
σ
)=
|
M
|−|
ρ
(
.
(3)
|
M
|
M
σ
)=0
M
σ
)=1
Therefore,
ρ
(
means that
σ
makes no reduction, and
ρ
(
implies that
no cases were selected from
M
using the
σ
function.
3.2 Evaluation Methodology
The final goal of the evaluation methodology is to obtain the best case selection method
for a given memory case for solving a particular problem. According to this vague
definition, we assume 3 main dimensions: the size of the case memory after the selection
process, the efficiency of the method, and the suitability of the case memory to solve
the problem.
In order to analyse these aspects we propose a 4-step methodology to evaluate case
selection algorithms:
Search WWH ::
Custom Search