Information Technology Reference
In-Depth Information
Fig. 12 FROC analysis for sclerotic metastasis CAD
The testing and training sets did not demonstrate a statistically significant difference
in sensitivity, with a bivariate chi-square test statistic of 3.7 (p = 0.055).
The etiology of false positive and false negative detections was evaluated. If a FP
was found to represent a true lesion, not marked by the radiologists creating the
reference standard set but detected by the CAD system, it was manually removed
from the FP statistic. There were a total of 15 true lesions detected by the CAD
system, not marked in the reference standard set. These lesions were not included in
the reference standard set because of small size and low attenuation in eight (53 %),
because of low attenuation and location at endplate in three (20 %), because of
small size in two (13 %), because the lesion was in an L5 with sacralization that was
thought to be outside the region of interest at ground truth marking in one (7 %),
and because of inadvertent deletion of one electronic lesion segmentation data
le
during creation of reference standard set in one (7 %). FP detections were most
often attributable to degenerative sclerosis (174 [28.1 %] of 620 actual detections)
and misclassi
cation of vertebral endplate bone cortex lying parallel to the (axial)
imaging plane (173 [27.9 %] of 620) (Table
4
). Other causes are noted in Table
4
.
There were 93 false-negative
findings, with 37 (40 %) caused by vertebral body
endplate proximity, 32 (34 %) caused by low attenuation, 17 (18 %) caused by
small size, and seven (8 %) caused by other reasons, such as the
finding was out of
the search region. Figure
13
shows a few examples.
Quantitative metrics were calculated for both the ground truth and computer-
detected lesions. The difference in mean lesion volume between ground truth and
computer-aided detections was not statistically signi
cant, with an approximate
volume difference of 0.3 % in the training set (t = 0.02, p = 0.987), and 0.9 %
difference in the testing set (t = 0.07, p = 0.943). The difference in mean lesion CT
attenuation (HU) between manually segmented lesions and computer-aided detec-
tions was not statistically signi
cant, with 8.0 % difference for the training set
(t = 2.7, p = 0.006) and 10 % for the testing set (t = 4.95, p < 0.0001).
