Analyzing the Results of Demonstration Studies - Evaluation Methods in Biomedical Informatics - page 229

TABLE 8.3. Hypothetical study results as a contingency
table.

Panel verdict, observed
results (no.)

System's prediction Disease No disease Total

Disease 27 (19.6) 14 (21.4) 41

No disease 16 (23.4) 33 (25.6) 49

Total 43 47 90

Numbers in parentheses are the expected results. Observed
results are in boldface.

false-positive rates may not be obtainable. This is because, in an alarm
system that continually monitors the value of one or more physiological
parameters, there is no way to count discrete true negative events.

Chi-Square Test for Statistical Significance
The basic test of statistical significance for 2 2 tables is performed by com-

puting the chi-square statistic. Chi-square can tell us the probability of com-
mitting type I errors (i.e., incorrectly inferring a difference when there is
none). Chi-square can be computed from the following formula:

¥

Â

2

( )

OE
E

-

i i

2 i

c =

i

where the summation is performed over all i cells of the table, O i is the
observed value of cell i and E i is the value of cell i expected by chance alone.
The expected values are computed by multiplying the relevant row and
column totals for each cell and dividing this number by the total number
of observations in the table. For example, Table 8.3 gives the results of a
hypothetical laboratory study of an information resource based on 90 test
cases. The columns give the gold standard verdict of a panel as to whether
each patient had the disease of interest, and the rows indicate whether the
patient was predicted by the system to have the disease of interest.
Observed results are in boldface type; expected frequencies for each cell,
given these observed results, are in parentheses.*
The value of chi-square for Table 8.3 is 9.8. A 2 2 contingency table is

associated with one so-called statistical degree of freedom. Intuitively, this
can be appreciated from the fact that, once the row and column totals for

¥

* The reader should confirm the calculations of expected values. For example, the
expected value for the disease-disease cell is obtained by multiplying the relevant
row total (41) by the relevant column total (43) and dividing the product by the
total number of participants (90).