Subjectivist Approaches to Evaluation - Evaluation Methods in Biomedical Informatics - page 252

These thoughts from the work of system designers alert us to the multiple
forces that shape the “effects” of introducing an information resource, the
unpredictable character of these forces, and the many viewpoints on these
effects that exist. These sentiments are highly consonant with the premises
underlying the subjectivist evaluation approaches.
Another connection is to the methodology of formal systems analysis,
generally accepted as an essential component of information resource
development. Systems analysis uses many methods that resemble closely
the subjectivist methods for evaluation that we introduce here. It is recog-
nized that systems analysis requires a process of information gathering
about the present system before a design for an improved future system
can be inferred. Systems analysis requires a process of information gather-
ing, heavily reliant on interviews with those who use the existing system in
various ways. Information gathering for systems analysis is typically por-
trayed as a cyclical, iterative process rather than a linear process. 3 In the lit-
erature of systems analysis we find admonitions, analogous to those made
by proponents of subjectivist evaluation, about an approach that is too
highly structured. An overly structured approach can misportray the capa-
bilities of workers in the system's environment, misportray the role of
informal communication in the work accomplished, underestimate the
prevalence of exceptions, and fail to account for political forces within every
organization that shape much of what happens. 4 Within the field of systems
analysis, then, there has developed an appreciation of some of the short-
comings of objectivist methods and the potential value of subjectivist
methods drawn from ethnography that we discuss here. 5
Also worthy of note is the high regard in which studies using subjectivist
methods are held when these studies are well conducted. In biomedicine,
one prominent example is Becker's 6 classic Boys in White. Another is
Bosk's 7 superb work, Forgive and Remember: Managing Medical Failure.
Several valuable studies in biomedical informatics are referenced in the fol-
lowing section.
Are Subjectivist Studies Useful in Informatics?
It is possible to argue that subjectivist approaches are applicable at all
stages of development of an information resource, but they are most clearly
applicable at two points in this continuum. First, as part of the design
process, a subjectivist study can document the need for the resource and
clarify its potential niche within a given work environment. 8,9 Indeed, it is
possible for system developers to misread or misinterpret the needs and
beliefs of potential users of an information resource 10,11 in ways that could
lead to failure of an entire project. Formal subjectivist methods, if applied
appropriately, can clarify these issues and direct resource development
toward a more valid understanding of user needs. There is already a sub-