Geoscience Reference
In-Depth Information
for accuracy assessment supposedly represent higher-quality data (i.e., less measurement error), so
these data may serve as a stand-alone basis for estimates of LC proportions and areas. Methods
for estimating area and proportion of area covered by the various LC classes have been developed
(Czaplewski and Catts, 1992; Walsh and Burk, 1993; Van Deusen, 1996). Recognizing this poten-
tially important use of reference data provides further rationale for implementing statistically
defensible probability sampling designs. This area estimation application extends to situations in
which LC proportions for small areas such as a watershed or county are of interest. A probability
sampling design provides a good foundation for implementing small-area estimation methods to
obtain the area proportions.
2.4 NONPROBABILITY SAMPLING
Because nonprobability sampling is often more convenient and less expensive, it is useful to
review some manifestations of this departure from a statistically rigorous approach. Restricting the
probability sample to areas near roads for convenient access or to homogeneous 3
3 pixel clusters
to reduce confounding of spatial and thematic error are two typical examples of nonprobability
sampling. A positive feature of both examples is that generalization to some population is statisti-
cally justified (e.g., the population of all locations conveniently accessible by road or all areas of
the map consisting of 3
¥
3 homogeneous pixel blocks). Extrapolation to the full map is problematic.
In the NLCD assessment, restricting the sample to 3
¥
3 homogeneous blocks would have repre-
sented roughly 33% of the map, and the overall accuracy for this homogeneous subset was about
10% higher than for the full map. Class-specific accuracies could increase by 10 to 20% for the
homogeneous areas relative to the full map.
Another prototypical nonprobability sampling design results when the inclusion probabilities
needed to meet the consistent estimation criterion of statistical rigor are unknown. Expert or
judgment samples, convenience samples (e.g., near roads, but not selected by a probability sampling
protocol), and complex,
¥
protocols are common examples. “Citizen participation” data
collection programs are another example in which data are usually not collected via a probability
sampling protocol, but rather are purposefully chosen because of proximity and ease of access to
the participants. This version of nonprobability sampling creates adverse conditions for statistically
defensible inference to any population. Peterson et al. (1999) demonstrate inference problems in
the particular case of a citizen-based, lake water-quality monitoring program. To support inference
from nonprobability samples, the options are to resort to a statistical model, or to simply claim
“the sample looks good.” In the former case, rarely are the model assumptions explicitly stated or
evaluated in accuracy assessment. The latter option is generally regarded as unacceptable, just as
it is unacceptable to reduce accuracy assessment to an “it looks good” judgment
ad hoc
.
Another use of nonprobability sampling is to select a relatively small number of sample sites
that are, based on expert judgment, representative of the population. In environmental monitoring,
these locations are referred to as “sentinel” sites, and they serve as an analogy to hand-picked
confidence sites in accuracy assessment. In both environmental monitoring and accuracy assess-
ment, judgment samples can play an invaluable role in understanding processes, and their role in
accuracy assessment for developing better classification techniques should be recognized. Although
nonprobability samples may serve as a useful initial check on gross quality of the data because
poorly classified areas may be identified quickly, caution must be exercised when a broad-based,
population-level description is desired (i.e., when the objective is to generalize from the sample).
Edwards (1998) emphasizes that the use of sentinel sites for population inference in environmental
monitoring is suspect. This concern is applicable to accuracy assessment as well.
More statistically formal approaches to nonprobability sampling have been proposed. In the
method of balanced sampling, selection of sample units is purposefully balanced on one or more
auxiliary variables known for the population (Royall and Eberhardt, 1975). For example, the sample
