Geoscience Reference
In-Depth Information
units in a category is divided by the total number of sample units of that category from the reference
data (i.e., the column total). This accuracy measure relates to the probability of a reference sample
unit's being correctly classified and is really a measure of omission error. This accuracy measure is
often called the
because the producer of the classification is interested in how
well a certain area can be classified. On the other hand, if the total number of correct sample units
in a category is divided by the total number of sample units that were classified into that category
on the map (i.e., the row total), then this result is a measure of commission error. This measure is
called
producer's accuracy
or reliability and is indicative of the probability that a sample unit classified
on the map actually represents that category on the ground (Story and Congalton, 1986).
user's accuracy
1.2.4
Discrete Multivariate Analysis
In addition to these descriptive techniques, an error matrix is an appropriate beginning for many
analytical statistical techniques, especially discrete multivariate techniques. Starting with Congalton
et al. (1983), discrete multivariate techniques have been used for performing statistical tests on the
classification accuracy of digital, remotely sensed data. Since that time many others have adopted
these techniques as the standard accuracy assessment tools (Rosenfield and Fitzpatrick-Lins, 1986;
Campbell, 1987; Hudson and Ramm, 1987; Lillesand and Kiefer, 1994).
One analytical step to perform once the error matrix has been built is to “normalize” or
standardize the matrix using a technique known as “MARGFIT” (Congalton et al., 1983). This
technique uses an iterative proportional fitting procedure that forces each row and column in the
matrix to sum to one. The rows and column totals are called marginals, hence the technique's name,
MARGFIT. In this way, differences in sample sizes used to generate the matrices are eliminated
and, therefore, individual cell values within the matrix are directly comparable. Also, because the
iterative process totals the rows and columns, the resulting normalized matrix is more indicative
of the off-diagonal cell values (i.e., the errors of omission and commission) than is the original
matrix. The major diagonal of the normalized matrix can be summed and divided by the total of
the entire matrix to compute a normalized overall accuracy.
A second discrete multivariate technique of use in accuracy assessment is called Kappa (Cohen,
1960). Kappa can be used as another measure of agreement or accuracy. Kappa values can range
from +1 to -1. However, since there should be a positive correlation between the remotely sensed
classification and the reference data, positive values are expected. Landis and Koch (1977) lumped
the possible ranges for Kappa into three groups: a value greater than 0.80 (i.e., 80%) represents
strong agreement; a value between 0.40 and 0.80 (i.e., 40%-80%) represents moderate agreement;
and a value below 0.40 (i.e., 40%) represents poor agreement.
The equations for computing Kappa can be found in Congalton et al. (1983), Rosenfield and
Fitzpatrick-Lins (1986), Hudson and Ramm (1987), and Congalton and Green (1999), to list just
a few. It should be noted that the Kappa equation assumes a multinomial sampling model and that
the variance is derived using the Delta method (Bishop et al., 1975).
The power of the Kappa analysis is that it provides two statistical tests of significance. Using
this technique, it is possible to test whether an individual land-cover (LC) map generated from
remotely sensed data is significantly better than a map generated by randomly assigning labels to
areas. The second test allows for the comparison of any two matrices to see whether they are
statistically, significantly different. In this way, it is possible to determine that one method/algo-
rithm/analyst is different from another one and, based on a chosen accuracy measure (e.g., overall
accuracy), to conclude which is better.
1.2.5
Sampling Size and Scheme
Sample size is another important consideration when assessing the accuracy of remotely sensed
data. Each sample point collected is expensive. Therefore, sample size must be kept to a minimum,
