Geoscience Reference
In-Depth Information
coincident with the classification process (feedback mechanism), (4) be consistent and repeatable,
and (5) be sufficiently robust in design to support subsequent change detection assessments.
The most common approach to classification assessment is through the analysis of confusion
matrices (Congalton, 1991). In this approach product classifications for a statistically robust number
of samples (
) are compared with “reference” data derived from an independent source (e.g.,
interpretation of aerial photography). The cost of “reference” data acquisition represents a signif-
icant challenge. This results in numerous limitations, which include: (1) only a small fraction of
the area of interest is used in the assessment process, (2) the content of a single confusion matrix
is used to characterize the accuracy of diverse areas (Zhu et al., 2000); (3) rare classes are frequently
underrepresented (
n
), and (4) accuracy characterization is limited to “macroscopic” levels (i.e.,
overall product and individual class levels).
Cost and logistics preclude highly detailed accuracy characterization based solely on conven-
tional ground reference data, and therefore one must investigate complementary, albeit indirect,
methods of accuracy assessment. This chapter describes an assessment strategy based on classifi-
cation consistency. For most land resources satellites (e.g., Landsat), extensive image overlap occurs
between scenes from adjacent World Reference System (WRS) frames. For a given adjacent
path/row pair, each scene provides a quasi-independent classification estimate of those pixels
resident in the overlap region. Intuitively, we would expect the level of classification agreement,
hereafter referred to as classification consistency, to be indicative of the absolute levels of classi-
fication accuracy (i.e., high levels of consistency should be associated with high levels of classifi-
cation accuracy).
The objectives here are to (1) establish a statistical link between classification consistency and
both user's and producer's accuracies, (2) develop an integrated accuracy assessment strategy to
quantify classification consistency and hence infer classification confidence, and (3) illustrate and
assess this approach using synoptic land-cover (LC) products.
n
10.2 LINK BETWEEN CLASSIFICATION CONSISTENCY
AND ACCURACY
To develop the statistical relationship between classification consistency for user's and pro-
ducer's accuracies, consider the case of two adjacent scenes, hereafter referred to as scenes number
1 and 2. If each scene is independently classified to a common scheme, the overlap region can be
used to quantify the classification consistency. For example, the consistency of class A in scene
number 1 can be written as:
M
Ê
Á
M
ˆ
˜
Â
Â
C
=
N P
P
N P
(10.1)
1
A
T
1
TA
2
TA
TTA
1
T
=
1
T
=
1
= the consistency, defined as the fraction of overlap pixels classed as A in scene number
1 that are also classed as A in scene number 2, M = the number of classes, P
where C
1A
= the probability
kTA
that a pixel of true class T is labeled as class A in scene number k, and N
= number of true class
T
T pixels in the overlap region. Note that P
is the producer accuracy of class T in scene k.
kTT
will be equal to the ratio of the number of correctly
classified class A pixels to the total number labeled as A:
The user accuracy for scene number 1
A
Ê
Á
M
ˆ
˜
Â
QNP
=
NP
(10.2)
1
A
A
1
AA
T
1
TA
T
=
1
