Geoscience Reference
In-Depth Information
the unequal inclusion probabilities. The SAS estimation procedures are designed to accommodate
these weights. Confidentiality of sample locations can be maintained because the necessary esti-
mation weights need not refer to any location information. The possibility exists that with the
location information stripped away, the data could be made available for limited general use for
applications requiring only the sample weights, and the map and reference labels. Users would
need to conduct their analyses via SAS or another software package that implements design-based
estimation procedures incorporating the sampling weights. Analyses ignoring this feature may
produce badly misleading results.
The use of SAS for accuracy assessment estimation provides two other advantages. SAS includes
estimation of standard errors as standard output. Standard error formulas are complex for the
sampling designs combining the advantages of both strata and clusters. Having available software
to compute these standard errors is highly beneficial relative to the alternative of writing one's own
variance estimation code and having to confirm its validity. Second, SAS readily accommodates
the fact that many accuracy estimates, for example producer's accuracy, are ratio estimators (i.e.,
ratios of two estimates). For ratio estimators, the SAS standard error estimation procedures employ
the common practice of using a Taylor Series approximation. The more complex design structures
that arise from more cost-effective assessments or use of existing data obtained from an ongoing
monitoring program will likely require more sophisticated analysis software than is available in
standard GIS and classification software. SAS does not provide everything that is needed, but its
capabilities represent a major step forward in computing for accuracy assessment analyses
.
2.6 PRACTICAL REALITIES OF SAMPLING DESIGN
In comments directed toward sampling design for environmental monitoring, Fuller (1999)
captured the essence of many of the issues facing sampling design for accuracy assessment. These
principles are restated, and in some cases paraphrased, to adapt them to accuracy assessment
sampling design: (1) every new approach sounds easier than it is to implement and analyze, (2)
more will be required of the data at the analysis stage than had been anticipated at the planning
stage, (3) objectives and priorities change over time, and (4) the budget will be insufficient.
2.6.1
Principle 1
Every new approach sounds easier than it is. Incorporating existing data for accuracy assessment
is a good case in point. While the data may be “free,” the analysis and research required to evaluate
the compatibility of the spatial units and classification scheme are not without costs. Confidentiality
agreements may need to be negotiated and strictly followed, spatial and temporal coverage of the
existing data may be incomplete and/or inadequate, and the response time for interaction with the
agency supplying the data may be slow because this use of their data may not be a top priority
among their responsibilities. Existing data that do not originate from a probability sampling protocol
are even more difficult to incorporate into a rigorous protocol and may be useful only as a qualitative
check of accuracy and to provide limited anecdotal, case-study information.
2.6.2
Principle 2
More will be required of the data at the analysis stage than had been anticipated at the planning
stage. This principle applies to estimating accuracy of subregions and other subsets of the data. That
is, a program designed for regional accuracy assessments will be asked to provide state-level estimates
and possibly even county-level estimates. Not only will overall accuracy be requested for these small
subregions, but also class-specific accuracy within the subregion will be seen as desirable informa-
tion. Accuracy estimates for other subsets of the data will become appealing. For example, are the
