Case-Based Reasoning in the Health Sciences: Foundations and Research Directions - Computational Intelligence in Healthcare: Advanced Methodologies - page 140

blood infections. Created by Ted Shortliffe, this knowledge-based system mapped
symptoms to diseases. It was clinically evaluated for effectiveness, and it led to
the development of the expert system shell EMYCIN. New generations of AI
systems in medicine expanded the range of AI methodologies in biomedical in-
formatics. Newer applications include: implementing clinical practice guidelines
in expert systems [70]; data mining to establish trends and associations among
symptoms, genetic information, and diagnoses; medical image interpretation;
and many more.
Impact on AI. Early researchers stressed the value of building systems for
testing AI methodologies. These systems provided valuable feedback to AI re-
searchers regarding the validity of their approaches. As reported by Shortliffe
and Buchanan, “Artificial intelligence, or AI, is largely an experimental science
- at least as much progress has been made by building and analyzing programs
as by examining theoretical questions. MYCIN is one of several well-known pro-
grams that embody some intelligence and provide data on the extent to which
intelligent behavior can be programmed. ... We believe that the whole field of
AI will benefit from such attempts to take a detailed retrospective look at ex-
periments, for in this way the scientific foundations of the field will gradually
be defined” [71]. When evaluating the advances of AI systems in medicine, sev-
eral levels of evaluation can be proposed, which can be roughly differentiated
as computer system, user satisfaction, process variables, and domain outcomes
levels:
1. The computer system level is how effectively the program performs its task.
Measures include diagnosis accuracy for a decision-support system providing
diagnostic recommendations, or precision and recall in an intelligent retrieval
system for medical information. Measures can be integrated in the system
programming.
2. The user satisfaction level involves assessing the user satisfaction with the
system - the user can be either the physician or the patient, whether the
patient uses the system or not. A questionnaire can be administered to the
patients or physicians.
3. The process variables level works by measuring some variable connected in
the clinical process, such as confidence in decision, pattern of care, adherence
to protocol, cost of care, and adverse effects [72].
4. The domain outcomes level aims to measure clinical outcomes of the sys-
tem. This requires conducting a randomized clinical trial to measure im-
provements in patient health or quality of life. For example, measures may
involve the number of complications, or the cost of care, or even the survival
duration.
Impact on the Health Sciences. Notably, critics of AI expressed concerns
that the field had not been able to demonstrate actual clinical outcomes. AI
researchers mostly showed satisfaction with computer system level evaluation
results, some user satisfaction level results and a few process variables results.