The Structure of Objectivist Studies - Evaluation Methods in Biomedical Informatics - page 86

After introducing some measurement terminology, this chapter formally
establishes the distinction between measurement studies designed to
explore with how much error “things of interest” in informatics can be mea-
sured, and demonstration studies , which apply these measurement proce-
dures to answer evaluation questions of substantive and practical concern.
The distinction between measurement and demonstration studies is more
than academic. As they are defined here, pure measurement studies are
rarely done in informatics. For example, a review of the literature on atti-
tudes toward information technology in health care, covering 11 years,
revealed only 17 articles that could be classified as reporting measurement
studies. 2 In the informatics literature, it appears that measurement issues
usually are embedded in, and often confounded with, demonstration issues.
Although attitudes pose some notoriously difficult challenges for mea-
surement, similar challenges exist across the full range of outcomes, as
introduced in Chapter 3, that are of concern in informatics. Indeed, a more-
recent review of objectivist studies of clinical information systems revealed
that only three of 27 published studies paid explicit attention to measure-
ment issues. 3 This matter is of substantial significance because deficiencies
in measurement can profoundly affect the conclusions drawn from a
demonstration study. The quote that begins this chapter alerts us to the fact
that our ability to investigate is circumscribed by our ability to measure.
Unless we possess or can develop ways to measure what is important to
know about our information resources, our ability to conduct evaluation
studies—at least those using objectivist approaches—is substantially
limited.
This chapter, then, lays the groundwork for understanding the interplay
between measurement and demonstration, a relationship that is developed
more deeply in the following four chapters on objectivist studies. The next
two chapters explore measurement issues in detail. The final two chapters
in this group focus on the design and conduct of demonstration studies.
Measurement Process and Terminology
In this section, some ground rules, definitions, and synonyms will be dis-
cussed that relate to the process of measurement. These definitions may use
some familiar words in unfamiliar ways, and the authors apologize for what
may appear to be an exercise in transforming the self-evident into the
obscure. The process of measurement and the interrelations of the terms to
be defined are illustrated in Figure 4.1.
Measurement
Measurement is the process of assigning a value corresponding to the pres-
ence, absence, or degree of a specific attribute in a specific object. The terms