Developing and Improving Measurement Methods - Evaluation Methods in Biomedical Informatics - page 174

item formats in terms of the quality of the measurement information
obtained. 22,26 Instrument designers choose formats largely based on the
goodness of fit to the attribute being assessed, and the mechanics of the
measurement process.
We now explore how multiple items can be used to form a scale. Table
6.4 contains an excerpt from a longer questionnaire that assesses the atti-
tudes of academic physicians toward information technology. 27 Each of the
items in Table 6.4 addresses the perceived effects of computers on a par-
ticular aspect of health care, but the items can be seen as having something
deeper in common. Each reflects, in part, a sense of optimism about the
future role of information technology in health care. The response options
form a bipolar axis. We might expect an individual who responds favorably
to one item to have a tendency to respond favorably to the other items in
the set because of this general belief or outlook. In this sense, each item can
be seen as an observation of the attribute “optimism.” The assumption can
be tested via an appropriate measurement study, and if the assumption
holds, a person's level of optimism may be assessed using the sum (or
average) of the responses to the set of items.* Across a set of items that
address the same underlying attribute, it is assumed that the idiosyncratic
reactions to the individual items cancel out and the average reflects the
individual's true belief.
We already know how to test such an assumption using a measurement
study, by examining the distributions of the responses to the individual
items and the correlations among them, after administering the items to a
representative sample of respondents. Perfect correlation among the dif-
ferent items is not expected. In this particular set of items, all but one of
the items were found to be well behaved. The reliability of the scale with
the poorly behaved item removed was 0.86. Of course, showing that the
items form a well-behaved cluster does not demonstrate that they combine
to assess optimism. Additional studies of the validity of the scale are
required for that purpose.
Scales to measure attitudes and beliefs are typically developed through
an iterative process where the investigators first clearly identify the
attribute to be assessed and the populations of respondents who will be
completing the ultimate form. They then create an initial set of items. To do
this, they might conduct open-ended interviews or focus groups, or develop
an initial item set from their own personal experience. The scale develop-
ers then conduct measurement studies, administering the scale to samples
of persons and identifying, revising, or replacing items that are not well
behaved. Over what is often a succession of measurement studies, the reli-
ability of the scale usually improves to acceptable levels. The validity of the
* Summing and averaging items yield equivalent results as long as the respondent
completes all the items composing the scale.