The Design of Demonstration Studies - Evaluation Methods in Biomedical Informatics - page 219

Self-Test 7.2
For each of the following short study scenarios, try to identify any of the
potential biases or threats to validity discussed in this chapter. Each sce-
nario may contain more than one bias or threat to validity. Then consider
how you might alter the resource implementation or evaluation plans to
reduce or quantify the problem.

1. As part of its initiative to improve patient flow and teamwork, a family
practice intends to install an electronic patient scheduling resource when it
moves to new premises in 3 months' time. Evaluators propose a 1-month
baseline study of patient waiting times and phone calls between clinicians,
starting at 2 months and conducted prior to the move, to be repeated imme-
diately after starting to use the new resource in 3 months.
2. A bacteriology laboratory is being overwhelmed with requests for
obscure tests with few relevant clinical data on the paper request forms. It
asks the hospital information system director to arrange for electronic
requesting and drafts a comprehensive three-screen list of questions clini-
cians must answer before submitting the request. The plan is to evaluate
the effects of electronic test ordering on appropriateness of requests by
randomizing patients to paper request forms or electronic requests for the
next year. The staff members intend to present their work at a bacteriology
conference.
3. A renowned chief cardiologist in a tertiary referral center on the West
coast is concerned about the investigation of some types of congenital heart
disease in patients in her unit. A medical informatics expert suggests that
her expertise could be represented as reminders about test ordering for the
junior staff looking after these patients. She agrees, announces her plans at
the next departmental meeting, and arranges system implementation and
training. Each patient is managed by only one junior staff member; there
are enough staff members to allow them to be randomized. After the trial,
the appropriateness of test ordering for each patient is judged by the chief
cardiologist from the entire medical record. It is markedly improved in
patients managed by the doctors who received reminders. Based on these
results, the hospital chief executive agrees to fund a start-up company to
disseminate the reminder system to all U.S. cardiology units.

Answers to Self-Tests

Self-Test 7.1
1. (a) Independent variables: time period before and after resource

installation (four levels).
(b) Dependent variables and measurement strategy: attitude to infor-
mation technology—30-item survey.