Environmental Engineering Reference
In-Depth Information
Technical Evaluation
The second type of evaluation consists of assessing whether components function
in technical terms and are integrated in such a way that the foreseen analyses can
be performed. The case studies played a central role in the technical evaluation by
providing an operational benchmark for the required properties of the individual
components as well as their integration in the framework.
Technical Evaluation of Procedures for Using the Integrated
Framework
As discussed before, the technical evaluation of the procedure was performed at the
same time as the conceptual evaluation but the technical evaluation focussed on
the operational feasibility of the procedure. It was assessed using criteria like the
distribution of roles among users, degree of interaction among policy experts and
modellers and time needed to complete an integrated assessment. These criteria
were made operational using a set of questions that were asked both to policy
experts and modellers at the end of the evaluation meeting provided these had not
yet been discussed spontaneously:
-
Does information required for problem formulation from policy experts match
their skills?
Are the guidelines provided for problem formulation easy to use by the modellers?
-
-
Does the translation of policy questions of the experts into experiments compatible
with SEAMLESS-IF match with the modellers' skills?
Does problem formulation with guidelines of the SEAMLESS-IF procedure
-
produce the necessary information to perform an assessment?
The evaluation revealed that a single meeting is not sufficient to define an IA
project in a computer-based tool such as SEAMLESS-IF. This was partly due to the
framework not being ready yet, hampering the assessment of whether specific
questions could be addressed by the framework, and partly due to its complexity.
Another key concern was the necessary level of knowledge of the modellers of
different disciplines in order to frame the policy question in a manner suitable for
assessment within the modelling chains. Another key reason to have more than one
meeting is the time needed for sharing a minimum level of mutual understanding
between policy experts and modellers.
Technical Evaluation of Quantitative Tools
In the case of quantitative tools the technical evaluation focussed first on
assessing the functioning of the individual components after which the combination
