Geoscience Reference
In-Depth Information
•
The need to perform a self-evaluation and to learn from your mistakes
•
The ability to compare method/algorithms/analysts quantitatively
•
The desire to use the resulting maps/spatial information in some decision-making process
There are many examples in the literature as well as an overwhelming selection of anecdotal
evidence to demonstrate the need for accuracy assessment. Many different groups have mapped
and/or quantified the amount of tropical deforestation occurring in South America or Southeast
Asia. Estimates have ranged by almost an order of magnitude. Which estimate is correct? Without
a valid accuracy assessment we may never know. Several federal, state, and local agencies have
created maps of wetlands in a county on the eastern shore of Maryland. Techniques used to make
these maps included satellite imagery, aerial photography (various scales and film types), and ground
sampling. Comparing the various maps yielded little agreement about where wetlands actually
existed. Without a valid accuracy assessment we may never know which of these maps to use.
It is no longer sufficient just to make a map using remotely sensed or other spatial data. It is
absolutely necessary to take some steps toward assessing the accuracy or validity of that map.
There are a number of ways to investigate the accuracy/error in spatial data including, but not
limited to, visual inspection, nonsite-specific analysis, generating difference images, error budget
analysis, and quantitative accuracy assessment.
The goal of this chapter is to review the current knowledge of accuracy assessment methods
and stimulate the reader to further the progression of diagnostic techniques and information to
support the appropriate application of spatial data. The ultimate objective is to motivate everyone
to conduct or demand an appropriate accuracy assessment or validation and make certain it is
included as an essential metadata element.
1.2 ACCURACY ASSESSMENT OVERVIEW
1.2.1
Historical Review
The history of accuracy assessment of digital, remotely sensed data is relatively short, beginning
in about 1975. Before 1975 maps derived from analog, remotely sensed data (i.e., photo interpre-
tation) were rarely subjected to any kind of quantitative accuracy assessment. Field checking was
typically performed as part of the interpretation process, but no overall map accuracy or other
quantitative measures of quality were typically incorporated into the analysis. Only after photo
interpretation began being used as reference data to compare maps derived from digital, remote
sensor data did issues concerning the accuracy of the photo interpretation arise. All the accuracy
assessment techniques mentioned in this chapter can be applied to assessing the accuracy of both
analog and digital remotely sensed data (Congalton and Mead, 1983; Congalton et al., 1983).
The history of accuracy assessment can be effectively divided into four developmental epochs
or ages. In the beginning, no real accuracy assessment was performed; rather, an “it-looks-good”
mentality prevailed. This approach is typical of many new, emerging technologies. Despite the
maturing of the technology over the last 25 years, some remote sensing analysts are still stuck in
this mentality. Of course, the map must “look good” before any further analysis should be per-
formed. Why assess a map that is obviously poor? However, while “looking good” is a required
characteristic, it is not sufficient for a valid assessment.
The second age of accuracy assessment could be called the epoch of nonsite-specific assessment.
During this period, overall acreages were compared between ground estimates and the map without
regard for location (Meyer et al., 1975). In some instances, such as imagery with very large pixels
(e.g., AVHRR imagery), a nonsite-specific assessment may be the best and/or only choice for
validation. For most imagery, the age of nonsite-specific assessment quickly gave way to the age
of the site-specific assessment (third age). In a site-specific assessment, actual places on the ground
