Markov Chain Monte Carlo Methods: Theory and Applications - Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications - page 71

3.3.6 Working with the Posterior Sample

After tuning the proposal distribution, assessing convergence, and ensuring insen-
sitivity to the Markov chain starting point, the end result of MCMC integration is
a sample of the target posterior PDF. As with any discrete sample, moments and
quantiles can be computed, and the maximum likelihood estimate (or maximum
a posteriori estimate; mode) can be obtained. In addition, while chain monitoring
and convergence diagnostics lend confidence that the algorithm has sampled a
stationary posterior distribution, it is also useful to present an estimate of the
error (termed the Monte Carlo error) in the posterior sample associated with
the discrete nature of the data. While it is impossible to precisely determine the
error if the exact posterior distribution is unknown, it is possible to approximate
it by examining the variance of the asymptotic distribution of, for example, the
mean of the distribution for increasing numbers of samples in the chain. For a
clear and comprehensive discussion of Monte Carlo error, the reader is referred
to Flegal et al. ( 2008 ). Summary statistics may obscure some of the relevant
features of the posterior sample (i.e., multi-modality), and it is common practice
to analyze the data from multiple different perspectives. Plots of one-dimensional
histograms of parameters x provide an initial indication of the center of mass and
dispersion in the posterior distribution, however, integration over the remaining

d 1
dimensions can mask inter-parameter relationships and multiple modes. For this
reason, it is common practice to examine two-dimensional marginals for every pair
of parameters. These are typically presented either as scatter plots or contour plots
of the posterior PDF. To obtain a more robust estimate of the posterior mode and
structure from the discrete sample, most studies apply a kernel density estimate
(KDE) to the posterior data ( Wand and Jones 1995 ; Tamminen and Kyrola 2001 ).
KDE consists of multiplying every data point by a kernel function (e.g., Gaussian)
with width determined from the sample ( Jones et al. 1996 ). The result is a smoothed
representation of the posterior sample that does not suffer from potential errors
introduced in the specification of histogram bin widths and locations.
Because MCMC affords near infinite flexibility in the specification of the PDFs
in ( 3.3 ), the posterior sample can be used to examine the error introduced by
approximations made in the implementation of simpler and/or more computation-
ally efficient posterior estimates (e.g., optimal estimation-type satellite retrievals;
Rodgers 2000 ). In applications that involve uncertainty quantification, the posterior
PDF represents the variability in a set of model output variables y associated with
changes in a set of input parameters x . The posterior PDF can thus be used to
examine the sensitivity of model output to changes in parameters, as well as the
relationships between parameters. The degree of sensitivity in a parameter or set of
parameters is directly related to the reduction in the dispersion of the prior PDF.
For parameters that exert large influence over the model state, a small change
in parameter values will produce a relatively large change in model output. As
such, the posterior PDF will narrow relative to the prior. The degree of sensitivity
can be formalized via computation of the Shannon Information content ( Shannon
and Weaver 1949 ; Rodgers 2000 ; Cooper et al. 2006 ), which is computed as the