Subjectivist Approaches to Evaluation - Evaluation Methods in Biomedical Informatics - page 263

• The investigators' adherence to these rules
• Accessibility of the data to others if necessary
• Value of the resulting studies to their respective audiences
We discussed how these factors apply to objectivist studies earlier in the
topic, and we have discussed them as they apply to subjectivist studies
to some extent in this chapter. A more thorough discussion is found in
Chapter 10.
Ultimately, each reader of this volume must make a personal judgment
about the credibility of any of the evaluation approaches presented. None
of the approaches is beyond challenge. We specifically caution the reader
against establishing an objectivist approach as a standard, and then assess-
ing subjectivist approaches using the specific characteristics of this standard.
This would inequitably frame the competition using the logic, definitions,
and assumptions unique to one of the competitors. For example, consider
the question of whether subjectivist approaches can establish causality
as well as their objectivist counterparts. If cause and effect are defined
as proponents of objectivist methods see the world, of course the answer is
no. (Objectivist work establishes cause and effect through randomization
and experimental control. Since subjectivist work does not employ ran-
domization and control, it cannot therefore establish cause and effect.)
The argument changes if cause and effect are defined more generically,
however. If both sides accept that one can establish cause and effect by
building a logical, believable case, they will conclude that both objectivist
and subjectivist approaches can approach such issues. They will just do it
differently.
It is also human nature to compare anything relatively new to an ideal-
ization of what is familiar. Because objectivist studies may be more famil-
iar, it is tempting to compare subjectivist methods against the perfect
objectivist study, which is never realized in practice. Every objectivist study
has limitations that are usually articulated at the end of a study report.
Many such reports end with a lengthy list of limitations and cautions and a
statement that further research is needed. For these reasons, rarely has any
one study, objectivist or subjectivist, ended a controversy over an issue of
scientific or social importance.
Two Example Abstracts
To convey both the substance and some of the style of subjectivist work in
informatics, we include below abstracts of two published studies. The first
is Forsythe's 10 1992 work, which had substantial impact on a project at the
design stage. The second is Aydin's 13 1989 work, which addressed the impact
of a deployed information resource.