Geoscience Reference
In-Depth Information
and (3) appropriate simplicity to implement and analyze (Stehman, 1999). These criteria hold
whether the reference data are crisp or fuzzy and will be prioritized differently for different
assessments. Because these criteria often lead to conflicting design choices, the ability to compro-
mise among criteria is a crucial element of the art of sampling design.
2.2 MEETING THE CHALLENGE OF COST-EFFECTIVE SAMPLING DESIGN
Effective sampling practice requires constructing a design that affords good precision while
keeping costs low. Strata and clusters are two basic sampling structures available in this regard,
and often both are desirable in accuracy assessment problems. Unfortunately, implementing a design
incorporating both features may be challenging. This topic will be addressed in the next subsection.
A second approach to enhance cost-effectiveness is to use existing data or data collected for purposes
other than accuracy assessment (e.g., for environmental monitoring). This topic is addressed in the
second subsection.
2.2.1
Strata vs. Clusters: The Cost vs. Precision Paradox
The objective of precise estimation of class-specific accuracy is a prime motivation for stratified
sampling. In the typical implementation of stratification in accuracy assessment, the mapped LC
classes define the strata, and the design is tailored to enhance precision of estimated user's accuracy
or commission error. Stratified sampling requires all pixels in the population to be identified with
a stratum. If the map is finished, stratifying by mapped LC class is readily accomplished. Geographic
stratification is also commonly used in accuracy assessment. It is motivated by an objective
specifying accuracy estimates for key geographic regions (e.g., an administrative unit such as a
state or an ecological unit such as an ecoregion), or by an objective specifying a spatially well-
distributed sample. It is possible, though rare, to stratify by the cross-classification of land-cover
class by geographic region. The drawback of this two-way stratification is that resources are
generally not sufficient to obtain an adequate sample size to estimate accuracy precisely in each
stratum (e.g., Edwards et al., 1998).
The rationale for cluster sampling is to obtain cost-effectiveness by sampling pixels in groups
defined by their spatial proximity. The decrease in the per-unit cost of each sample pixel achieved
by cluster sampling may result in more precise accuracy estimates depending on the spatial pattern
of classification error. Cluster sampling is a means by which to obtain spatial control (distribution)
over the sample. This spatial control can occur at two scales, termed regional and local. Regional
spatial control refers to limiting the macro-scale spatial distribution of the sample, whereas local
spatial control reflects the logical consequence that sampling several spatially proximate pixels
requires little additional effort beyond that needed to sample a single pixel. Examples of clusters
achieving regional control over the spatial distribution of the sample include a county, quarter-
quad, or 6-
6-km area. Examples of design structures used to implement local control include
blocks of pixels (e.g., 3
¥
5 pixel blocks), polygons of homogeneous LC, or linear clusters
of pixels. Both regional and local controls are designed to reduce costs, and for either option the
assessment unit is still an individual pixel.
Regional spatial control is designed to control travel costs or reference data material costs. For
example, if the reference data consist of interpreted aerial photography, restricting the sample to a
relatively small number of photos will reduce cost. If the reference data are collected by ground
visit, regional control can limit travel to within a much smaller total area (e.g., within a sample of
counties or 6-
¥
3 or 5
¥
6-km blocks). When used
alone, local spatial control may not achieve these cost advantages. For example, a simple random
or systematic sample of 3
¥
6-km blocks, rather than among all counties or 6-
¥
3 pixel blocks providing local spatial control may be widely dispersed
across the landscape, therefore requiring many photos or extensive travel to reach the sample clusters.
¥
