Quantitative Sleep Monitoring - Quantitative EEG Analysis Methods and Clinical Applications - page 284

problem of REM detection. In this case, the classifier needs to decide only whether
the segment contains REMs or not:

TN

K = (10.9)

i

TN + FP

i i

It should be noted that either the specificity or the sensitivity alone is not a suffi-
cient measure of goodness. Consider a classification algorithm that always classified
the point as being in X i . Such a method would produce a sensitivity of 1 and a speci-
ficity of 0. Similarly, an algorithm that always classified the point as not being in X i
would produce a specificity of 1 and a sensitivity of 0. The ideal algorithm would
produce a value of 1 for each, so some combination of the two should be used when
creating a performance metric.
When applied to sleep detection there are five different categories. A data point
can be from either the wake state, NREM stage 1, NREM stage 2, NREM stages
3/4, or REM sleep. The above method would then yield five different specificity and
sensitivity values. A performance metric could be created by using any one of the 10
values (such as the minimum or maximum) or some combination of the 10 values
(such as the average).

10.16.2 Contingency Table
Another method used to evaluate the performance of a multicategory classification
technique is a contingency table. This is used specifically for sleep detection by Estévez
[45] and Sinha [57]. A contingency table indicates the relationship between two or
more variables. In this case the variables are the classifications given to the data points
by the technique and the classifications given to the same data points by an expert (the
polysomnographer). A contingency table would be similar to the one shown in Table
10.9. Each cell of the table represents the number of points that the system and the
expert classified under the respective row and column. The cell value at the intersec-
tion of the NREM-I column and the WAKE row indicates that there were 11 points
that the expert classified as being from NREM stage 1 sleep and the system classified
as being from the wake state. In a contingency table the cells along the main diagonal
indicate correct classifications, and all other cells are incorrect classifications.
In addition to the contingency table, Estévez also extracts further information
from the contingency table such as the kappa index. The kappa index compares the

Table 10.9 Example of What a Contingency Table Might Look Like

Expert's Classifications

System's
Classifications

NREM-III
& IV WAKE NREM-I NREM-II REM Total

WAKE 543 11 18 6 3 581

NREM-I 23 456 6 9 22 516

NREM-II 21 21 348 52 21 463

NREM-III and IV 43 12 2 632 43 732

REM 12 7 12 23 312 366

Total 642 507 386 722 401 2,658