Information Technology Reference
In-Depth Information
Fig. 6.10 A4
×
4 checkerboard dataset with 400 instances (100 instances in the
minority class: dots). Dotted lines are for visualization purpose only.
Tabl e 6 . 5
The datasets of the experiments reported in [215].
CB2
×
2(50) CB2
×
2(25) CB2
×
2(10) CB4
×
4(50) CB4
×
4(25) CB4
×
4(10)
No. cases
200
400
1000
200
400
1000
No. feat.
2
2
2
2
2
2
Cl. Hea. D.
Diabetes
Ionosphere
Liver
Sonar
Wdbc
No. cases
207
768
351
345
208
569
No. feat.
13
8
34
6
60
30
The performance assessment was based on 20 runs of the holdout method
and computation of the following statistics (all in percentage) in each run:
Correct classification rate:
P
ct
=1
P
et
,with
P
et
defined as in Sect. 3.2.2.
Balanced correct classification rate,
P
bct
,avariantof
P
ct
computed with
the equal prior assumption.
Area under the Receiver Operating Characteristic curve, which measures
the trade-off between sensitivity and specificity in two-class decision tables
(for details, see e.g. [149]). The higher the area the better the decision rule.
−
The last two performance measures are especially suitable for unbalanced
datasets, such as the artificial checkerboard datasets, where an optimistically
high
P
ct
could arise from a too high sensitivity or specificity.
P
ct
was only
used for the real-world datasets.
The mean values of
P
ct
and
P
bct
(in the 20 runs) with the standard de-
viations are shown in Table 6.6. The highest mean value (a "win") appears
in bold. The Friedman test and the multiple sign test, adequate for multi-
comparison assessment of classifiers [51, 79, 102, 195], found no statistically
