Developing and Improving Measurement Methods - Evaluation Methods in Biomedical Informatics - page 152

TABLE 6.3. Correlations between observations for the
“typical measurement result”.
Item 1 2 3 4 5

1 — 0.41 - 0.30 0.89 0.91

2 — 0.49 0.73 0.74

3 — - 0.13 0.09

4 — 0.94

coefficient of 0.62, as shown in the bottom row of Table 6.1, is computed
from these four ordered pairs.
The other corrected part-whole correlations are computed by creating
analogous ordered pairs for each of the other observations: the scores on
Observation 2 paired with the total scores excluding Observation 2, which
yields a correlation of 0.83; the scores on Observation 3 paired with the
total scores excluding Observation 3, which yields a correlation of 0.11; and
so on. These calculations are relatively straightforward on a spreadsheet.
Having seen how the short-cut method of part-whole correlations works
in practice, we now explore the full matrix of correlations among all pairs of
observations. Table 6.3 displays these correlations for the measurement
results given in Table 6.1. Overall, there are 5(5 1)/2, or 10, correlation coef-

ficients to inspect. The correlation between observations i and j is found at
the intersection of the i th row and j th column. Because this matrix of inter-
correlations is symmetrical about the diagonal, and the diagonal elements
are equal to 1.0, only the values of elements above the diagonal are shown.
In this example, Observation 3 is not well behaved. This is seen several
ways. From Table 6.1, it can be seen that an object with the highest total
score (A) has a relatively low score on Observation 3. The object with the
second lowest total score (B) has the highest score on Observation 3. We
also see evidence of Observation 3's misbehavior because the corrected
part-whole coefficient is less than 0.4 whereas the others are high. In Table
6.3 the correlations between Observation 3 and the other observations, seen
in boldface type, show no consistent pattern: two are negative, and two are
positive. Deleting Observation 3 increases the reliability of measurement
(in this case, from 0.81 to 0.89), even though the number of observations in
the measurement process is decreased. Observations can also fail to be well
behaved if their mean values (across all objects observed) are close to either
the high or low extremes, or if they display no variability across objects.
Such observations add no useful information to the measurement process
and should be modified or deleted.

-

What Corrected Part-Whole Correlations Can Reveal
In later sections of this chapter we discuss specific ways to improve mea-
surement technique for specific situations that arise frequently in infor-