Biomedical Engineering Reference
In-Depth Information
TABLE 5.1. Perfectly reliable measurement.
Observations
bject
1
2
3
4
5
Object score
A
3
3
3
3
3
3
B
4
4
4
4
4
4
C
2
2
2
2
2
2
D
5
5
5
5
5
5
The total variability is the statistical variance of all observations over all
objects used in the measurement study, and the variability due to error can
be estimated as the extent to which the results of individual observations
vary within each object. Returning to the archery metaphor,
V
error
can be
likened to the directly observable scatter of the arrows the archer(s) fire at
each target. The greater the scatter, the lower the reliability.
As a first example, consider the following result of a measurement study
(Table 5.1), a result rarely seen in nature. Each object displays an identical
result for all five observations thought to comprise a scale. The objects in
this example could be clinical cases, and the observations could be the
ratings by expert judges of the quality of care provided in each case. Alter-
natively, the objects could be people, and the observations could be their
responses on a questionnaire to a set of questions that address a specific
attribute. Table 5.1 is the first example of an objects-by-observations matrix,
which is the way results of measurement studies are typically portrayed.
Because scores for each object are identical across observations, the best
estimate of the error is 0 and the best estimate of the reliability, denoted as
r
, is 1.0. (In this situation, each arrow always lands in the same place on
each target!) Because the average of the observations is the result of the
measurement, Object A's score would be 3, Object B's score would be 4,
and so forth. There is no scatter or variability in the results from the indi-
vidual observations, so we place high confidence in these results as a mea-
surement of
something.
(A separate issue is whether this
something
is in fact
what the investigator thinks it is. This is the issue of validity, discussed later.)
If the matrix in Table 5.1 were the result of a measurement study, this esti-
mate of perfect reliability would generalize only to the real or hypotheti-
cal population of objects from which the specific objects employed in the
study were sampled. Because the number of objects in this particular
example is small, we should place low credence in a belief that the mea-
surement process has perfect reliability. If just one additional object were
added to the sample in the measurement study, and for that object the
results of all five observations were not identical, this modification in the
measurement study results would substantially lower the estimated
reliability.
In a more typical example (Table 5.2), the results of the observations vary
within each object. Each object's score is now the average of observations
Search WWH ::
Custom Search