Biomedical Engineering Reference
In-Depth Information
FIGURE 5.7. Hypercritic study.
judges did not always agree, and that the judges did not always agree among
themselves. Comment 1 was generated by Hypercritic—and perhaps by
one or more judges as well—on the initial review of the patient's record
and was subsequently rated as correct by judges A, B, and D. Comment 2
was not generated by Hypercritic. On the initial review it was generated by
one or more of the judges, and it was subsequently endorsed by judges B,
C, D, and G. A demonstration study exploring the accuracy of Hypercritic's
advice would seek to correlate the level of judges' endorsement of these
comments on the second review with Hypercritic's “endorsement,” as
inferred from whether Hypercritic generated the comment on the initial
review. We can consider the pooled ratings of the judges to be a less-than-
TABLE 5.5. Results from a subset of 12 comments in the Hypercritic study.
Generated by
Ratings by each judge
Judges'
Hypercritic?
ABCDEFGH
“correctness” score
Comment no.
1
s
11010000
3
2
o
01110010
4
3
o
11110010
5
4
s
11010011
5
5
o
11111001
6
6
o
01110011
5
7
o
11110011
6
8
o
11110011
6
9
s
11011111
7
0
s
11111011
7
1
o
11111101
7
2
s
11011111
7
Search WWH ::
Custom Search