The Design of Demonstration Studies - Evaluation Methods in Biomedical Informatics - page 201

inference is stronger if the control doctors worked in the same wards during
the period the resource was introduced, and if similar kinds of patients,
subject to the same nonspecific influences, were being operated on during
the whole time period.
Even though the controls in this example are now simultaneous, skeptics
may still refute our argument by claiming that there is some systematic
unknown difference between the clinicians or patients in the reminder and
control groups. For example, if the two groups comprised the patients and
clinicians in two adjacent wards, the difference in the infection rates could
be attributable to systematic or chance differences between the wards.
Perhaps hospital staffing levels improved in some wards but not others, or
there was cross-infection by a multiply resistant organism but only among
patients in the control ward. To overcome such criticisms, we could expand
the study to include all wards in the hospital—or even other hospitals—but
this requires many more measurements, which would clearly take consid-
erable resources. Such externally and internally controlled before-after
studies are described later. We could try to measure everything that
happens to every patient in both wards and build complete psychological
profiles of all staff to rule out systematic differences, but we are still vul-
nerable to the accusation that something we did not measure—did not even
know about—explains the difference between the two wards. A better strat-
egy is to ensure that the controls are truly comparable by randomizing
them.
Simultaneous Randomized Controls
The crucial problem in the previous example is that, although the controls
were simultaneous, there may have been systematic, unmeasured differ-
ences between the participants in the control group and the participants
receiving the intervention. A simple, effective way to remove systematic dif-
ferences, whether due to known or unknown factors, is to randomize the
assignment of participants to control or intervention groups. Thus we could
randomly allocate half of the doctors on both wards to receive the antibi-
otic reminders, and the remaining doctors could work normally. We would
then measure and compare postoperative infection rate in patients
managed by doctors in the reminder and control groups. Providing that the
doctors never look after one another's patients, any difference that is sta-
tistically significant can reliably be attributed to the reminders, as the only
way other differences could have emerged is by chance. We discuss the
concept of statistical significance in the following chapter.
Table 7.6 shows the hypothesized results of such a study. The baseline
infection rates in the patients managed by the two groups of doctors are
similar, as would be expected because they were allocated to the groups by
chance. There is a greater reduction in infection rate in patients of reminder
physicians than those treated by the control physicians. The only systematic