Determining What to Study - Evaluation Methods in Biomedical Informatics - page 61

4. A biomedical informatics expert is asked for her opinion about a Ph.D.
project. She requests copies of the student's code and documentation for
review.
5. A new intensive care unit system is implemented alongside manual
paper charting for a month. At the end of this time, the quality of the com-
puterized data and data recorded on the paper charts is compared. A panel
of intensivists is asked to identify, independently, episodes of hypotension
from each data set.
6. A biomedical informatics professor is invited to join the steering
group for a clinical workstation project in a local hospital. The only docu-
mentation available to critique at the first meeting is a statement of the
project goal, a description of the planned development method, and the
advertisements and job descriptions for team members.
7. Developers invite educationalists to test a prototype of a computer-
aided learning system as part of a user-centered design workshop.
8. A program is devised that generates a predicted 24-hour blood glucose
profile using seven clinical parameters. Another program uses this profile
and other patient data to advise on insulin dosages. Diabetologists are asked
to prescribe insulin for the patient given the 24-hour profile alone, and then
again after seeing the computer-generated advice. They also are asked their
opinion of the advice.
9. A program to generate alerts to prevent drug interactions is installed
in a geriatric clinic that already has a computer-based medical record
system. Rates of clinically significant drug interactions are compared before
and after installation of the alerting program.
Four Evaluation Scenarios
In this section, four scenarios are introduced that collectively capture many
of the dilemmas facing readers of the topic.
1. A health problem or opportunity has been identified that seems
amenable to an information/communication resource, but there is a need
to define the problem in more detail.
2. A prototype information resource has been developed, but its usability
and potential for benefit need to be assessed prior to deployment.
3. A locally developed information resource has been deployed within an
organization, but no one really knows how useful it is proving to be.
4. A commercial resource has been deployed across a large enterprise, and
there is need to understand what impact it has on users, as well as on the
organization.
These scenarios do not address the full scope of evaluations in biomed-
ical informatics, but they cover much of what people do. For each, sets of