Analyzing the Results of Demonstration Studies - Evaluation Methods in Biomedical Informatics - page 240

2. The analysis example discussed above pertains only to a completely
randomized factorial design. The ANOVA methods employed for other
designs, including nested and repeated measures designs, require special
variants on this example.
3. Appropriate use of ANOVA requires that the measured values of the
dependent variables are distributed roughly according to a “normal” dis-
tribution, and also meet other statistical requirements. If the dependent
variables as measured fail to meet the assumptions, corrective actions such
as transformations of the data may be required, or ANOVA methods may
not be applicable.

Self-Test 8.2
1. Given below are the data from the Hypercritic study discussed in
Chapter 5. For these data, compute (a) Hypercritic's accuracy, sensitivity,
and specificity; (b) the value of chi-square; and (c) the value of Cohen's k .

Pooled rating by judges
Comment valid

Comment not valid

Hypercritic (≥5 judges) (<5 judges) Total

Comment generated 145 24 169

Comment not generated 55 74 129

Total 200 98 298

2. Review scenario 3 in Self-Test 7.1. Hypothetical results of that study
are summarized in the two tables below. The first table gives means and
standard deviations of the outcome measure, charges per patient, for each
cell of the experiment. Note that n = 6 for each cell. Interpret these results.

Advice mode
Advice always

Advice when

Hospital provided requested No advice

A 58.8 ± 7.9 54.8 ± 5.8 67.3 ± 5.6

B 55.2 ± 7.4 56.0 ± 4.7 66.0 ± 7.5

The second table gives the ANOVA results.

Source Sum-of-squares df Mean square F ratio p value

Main effects
Hospital 12.250 1 12.250 0.279 .601

Advice mode 901.056 2 450.528 10.266 < .001

Interaction
Hospital
by group 30.500 2 15.250 0.348 .709

Error 1316.500