Agriculture Reference
In-Depth Information
11.6 The Benchmarking Problem
Model-based SAE depends on models that are usually difficult to validate. If the
model is misspecified, the resulting predictors may perform poorly. Benchmarking
techniques should be used to verify these estimators.
Stakeholders and general users expect that the disseminated SA estimates are
coherent and consistent when compared with published official statistics. Coher-
ence is defined by the Australian Bureau of Statistics as
“The internal consistency of a statistical collection, product or release, as well as its
comparability with other sources of information, within a broad analytical framework
and over time”
SA estimates that are not consistent will obviously lose credibility with users.
Besides, aggregated model-based SAEs must correspond to the direct survey
estimate for a larger area (for example, at national level). This issue is very
important if the sample size for the larger area is sufficient for providing reliable
estimates, and if the direct estimate for the larger area has an official nature. If the
model-based SAE aggregation significantly departs from the corresponding direct
estimate for a large area, it suggests model failure.
These considerations corroborate the use of benchmarking, which is a form of
calibration that adapts the individual area level estimates so that their aggregation is
equal to a direct estimate for a large area (Pfeffermann and Tiller
2006
;
Pfeffermann
2013
).
More formally, assuming that the aggregation process involves all the areas, the
benchmarking equation can be defined in a general form as
X
D
g
j
ʸ
j
,
model
¼
X
D
g
j
ʸ
j
,
design
:
ð
11
:
48
Þ
j
¼1
j
¼1
The coefficients (
g
j
) are fixed weights such that
X
D
g
j
ʸ
j
,
design
are a design consistent
j
¼1
estimator of the total. Eq. (
11.48
) forces the model-based predictors to match a
design-based estimator over an aggregation of the areas for which the design-based
estimator is reliable.
For example, the model-based predictors of the total agricultural sector employ-
ment in Italian provinces should match the design-based estimate of the total
agricultural sector employment in Italy (that represents the sum of the design-
based estimate of Italian provinces), which can be considered accurate.
We can clarify the problem using a simple example described in Pfeffermann
and Tiller (
2006
). The US Bureau of Labor Statistics (BLS) uses state-space time
series models to provide monthly estimates concerning the employment in 9 Census
Divisions (CDs), the 50 states, and the District of Columbia. The models are fitted
to the direct sample estimates obtained from the Current Population Survey (CPS).
These models are needed because the samples from the CPS for each state and
each CD are too small to produce reliable estimates. Therefore, SAE techniques
Search WWH ::
Custom Search