Geoscience Reference
In-Depth Information
(including citizens' juries, community advisory committees, and mediation), while
Rowe and Frewer (2000) include different methods (including consensus confer-
ences and focus groups) in a similar generic evaluation based on criteria closely
matching those listed above. Such generic analyses show that each approach has its
own strengths and weaknesses, as illustrated by Petts' (2001) comparative analysis
of community advisory committees and citizens' juries in UK waste management
planning. The latter were more 'representative' of local publics while the former
provided greater learning opportunities over longer time scales.
An emphasis on procedural evaluation assumes that better participatory pro-
cesses lead to better environmental (and other) outcomes, yet there can, by defi ni-
tion, be no guarantee that this is the case (Munton, 2003). Very few existing studies
consider outcomes, and fewer still the relationship between process and outcome.
There are considerable methodological diffi culties to be overcome in tracking emer-
gent outcomes in the longer-term and detecting cause-effect relationships after an
event. There are also conceptual challenges in defi ning 'outcomes' and then somehow
measuring them, as well as institutional pressures for early evaluations to demon-
strate process effi cacy (Rowe and Frewer, 2004). Burgess and Chilvers (2006) dif-
ferentiate between
outputs
, the immediate substantive products of participatory
processes such as reports, assessments, and policy recommendations, and
outcomes
,
the emergent impacts and resulting changes such as improvements in environmental
quality, sustainability, social capital, individual/institutional learning, refl exivity and
behaviour change (see fi gure 24.1). While Beierle and Konisky (2001) offer an
optimistic assessment of environmental outcomes resulting from stakeholder engage-
ment in the North American Great Lakes region, Bickerstaff and Walker (2005)
provide a more pessimistic evaluation of two deliberative processes in UK local
transport planning which had little substantive impact on policy outcomes due to
institutional and national constraints (a fi nding shared by other retrospective evalu-
ations, such as Goodwin, 1998; Davies, 2002).
Evaluating relations between participatory processes and outcomes poses signifi -
cant methodological challenges. There is a pressing need for well designed longitu-
dinal research involving retrospective and 'real time' studies that more effectively
capture process dynamics and track emergent outcomes (Owens et al., 2004). The
purpose of early and ongoing evaluation is not only summative; it should also play
a formative role in shaping ongoing process design and enhancing refl exivity. Of
particular importance are features such as learning that straddle the distinction
between process and outcome and span individual through to institutional levels.
Sophisticated research designs are required to establish whether and how participa-
tory deliberation leads to transformations in participants' identities, knowledge,
values, and competencies (Petts, 1997; Davies and Burgess, 2004) as well as changes
in their environmental behaviour and action (Owens, 2000). If the claims of better
environmental decisions are to be properly tested there is also a need to monitor
and assess material environmental changes that result from participatory policymak-
ing and appraisal processes. Here is another opportunity for the interdisciplinarity
of environmental geography to make its mark. We must not forget, however, that
evaluation comes with its own politics and sensitivities - between the evaluator, the
evaluated, and wider audiences - which intensify as processes become larger in scale
and more high profi le (see Rowe et al., 2005). It is also important to recognise that
evaluation can itself become wrapped up in a cycle of decision justifi cation and be
used by decision institutions for instrumental purposes.
