Graphics Reference
In-Depth Information
Fig. 2.3 5 × 2-fold process
2.1.2 Performance Measures
Most of the preprocessing stages aim to improve the quality of the data. Such improve-
ment is later measured by analyzing the model constructed over the data and it
depends on the type of the DM process carried out afterwards. Predictive processes
like classification and regression rely in a measure of how well the model fits the
data, resulting in a series of measures that work over the predictions made.
In classification literature we can observe that most of the performance measures
are designed for binary-class problems [ 25 ]. Well-known accuracy measures for
binary-class problems are: classification rate , precision, sensitivity, specificity, G-
mean [ 3 ], F-score, AUC [ 14 ], Youden's index
[ 31 ] and Cohen's Kappa [ 4 ].
Some of the two-class accuracy measures have been adapted for multi-class prob-
lems. For example, in a recent paper [ 18 ], the authors propose an approximating
multi-class ROC analysis, which is theoretically possible but its computation is still
restrictive. Only two measures are widely used because of their simplicity and suc-
cessful application when the number of classes is large enough. We refer to classifi-
cation rate and Cohen's kappa measures, which will be explained in the following.
γ
Classification rate (also known as accuracy) : is the number of successful hits
relative to the total number of classifications. It has been by far the most commonly
used metric for assessing the performance of classifiers for years [ 1 , 19 , 28 ].
Cohen's kappa : is an alternative to classification rate, amethod, known for decades,
that compensates for random hits (Cohen 1960). Its original purpose was to mea-
sure the degree of agreement or disagreement between two people observing the
 
Search WWH ::




Custom Search