Environmental Engineering Reference
In-Depth Information
Conceptual Evaluation of the Procedure for Using
the Integrated Framework
Integrated assessment implies an analysis of policies from different perspectives or
disciplines, requiring cooperation between actors with different background.
Procedures for implementing policy assessments with the computer-based tools aid
this cooperation by structuring the exchange of ideas in such a way that policies can
be translated into scenarios that can be assessed by the available modelling chains.
The development of a procedure for using the framework is a way to ensure that
the integrated framework will be able to address the issues of interest for end-users.
It is thus closely linked to the aim of a conceptual evaluation (checking that design
of components and the integrated framework will lead to the desired functionalities
for end-users).
Evaluation criteria can be of very diverse abstraction levels. They can be a priori
defined or elaborated during the evaluation process (Darses et al.
2004
; Darses
2002)
. For the conceptual evaluation of the IA procedure only criteria of high
abstraction level were pre-defined, like relevance, understanding, etc. Commonly
procedures are assessed by comparing users' organisational situations and working
methods to spot possible incompatibilities without involving the users and assuming
sufficient knowledge of their working environment. However, to ensure the best
possible fit between the procedure, and more generally the development of
SEAMLESS-IF, and end-user requirements, potential end-users and modellers
from different disciplines were involved in the evaluation of the procedure.
Involving the end-users allowed inclusion of more concrete evaluation criteria
during the evaluation itself. Similar to the argument made with the test cases,
care has to be taken to ensure representativeness of the end-users involved in the
evaluation for the targeted end-users of the framework. Else there is a risk of gearing
the development of the framework to specific users in such a way that functionalities
for other users become limited.
SEAMLESS-IF is foreseen to be used within a participatory process involving
policy experts who bring the problem to study and integrative modellers who set up
the IA within SEAMLESS-IF. PE are either in charge of policy design or interested
in impacts of policies designed by others. A specific procedure has been developed
in which the framework use is a component of the impact assessment procedures of
the end-users (Fig.
10.2
). The later can include additional tools before or after the
scenario analysis done through SEAMLESS-IF (see, for example, SEC
2005
for the
EU procedure).
The SEAMLESS-IF procedure consists of three main steps: pre-modelling
(describing the problem and defining indicators at levels and for domains relevant
to the policy question at hand), modelling (selecting the appropriate models and
defining scenarios needed to assess the policy question at hand), and post-modelling
(analyzing and presenting the model results) (Thérond et al.
2009)
. This SEAMLESS-IF
procedure had a strong influence on the design of the GUI of SEAMLESS-IF, the
evaluation of which will be discussed below.
