Evaluation as a Field - Evaluation Methods in Biomedical Informatics - page 30

• Through these kinds of comparisons, it is possible to prove beyond rea-
sonable doubt that a resource is superior to what it replaced or to some
competing resource.

Chapters 4 through 8 of this topic explore these issues and address objec-
tivist methods in detail.
Contrast the above with a set of assumptions that derive from an
“intuitionist-pluralist” worldview that gives rise to a set of subjectivist
approaches to evaluation.

• What is observed about a resource depends in fundamental ways on the
observer. Different observers of the same phenomenon might legiti-
mately come to different conclusions. Both can be “objective” in their
appraisals even if they do not agree. It is not necessary that one is right
and the other wrong.
• It does not make sense to speak of the attributes of a resource without
considering its context. The value of a resource emerges through study
of the resource as it functions in a particular patient-care scientific or edu-
cational environment.
• Individuals and groups can legitimately hold different perspectives on
what constitutes desirable outcomes of introducing a resource into an
environment. There is no reason to expect them to agree, and it may be
counterproductive to try to lead them to consensus. An important aspect
of an evaluation would be to document the ways in which they disagree.
• Verbal description can be highly illuminating. Qualitative data are valu-
able in and of themselves and can lead to conclusions as convincing as
those drawn from quantitative data. Therefore, the value of qualitative
data goes far beyond that of identifying issues for later “precise” explo-
ration using quantitative methods.
• Evaluation should be viewed as an exercise in argument, rather than
demonstration, because any study, as House 4 points out (p. 72), appears

equivocal when subjected to serious scrutiny.

The approaches to evaluation that derive from this subjectivist philo-
sophical perspective may seem strange, imprecise, and “unscientific” when
considered for the first time. This stems in large part from widespread
acceptance of the objectivist world-view in biomedicine. The importance
and utility of subjectivist approaches to evaluation are emerging, how-
ever. Within biomedical informatics, there is growing support for such
approaches. 8-10 As has been stated previously, the evaluation mindset
includes methodological eclecticism. It is important for those trained in
classical experimental methods at least to understand, and possibly even to
embrace, the subjectivist worldview if they are going to conduct fully infor-
mative evaluation studies. Chapters 9 and 10 of this topic address subjec-
tivist approaches in detail.