Environmental Engineering Reference
In-Depth Information
based on iterative cycles of design and evaluation of the framework's prototypes.
This involved potential users for some aspects and in any cases, feedbacks between
'testers' and developers. Prime users (i.e. Directorate General of European Commission)
were involved in these cycles by the way of a User Forum (van Ittersum et al.
2008)
and other potential users (regional and national decision makers, stakeholder
groups) have been periodically consulted. Starting from individual components the
first phase consists of designing the structure of the integrated framework (for
example by making mock-ups showing how the final system may look without any
functionality). Conceptual evaluation checks that the design of components and the
integrated framework will lead to the desired functionalities for end-users. This
phase precede the 'verification phase' as identified by Sojda
(2007)
with explicit
attention to end-user requirements. It aims at defining and clarifying the operational
objectives of the tool development (i.e. main expected capacities) and the future
tool shape (e.g.: main functionalities and material form) (Dieste et al.
2003)
.
The second phase consists of building the framework from the individual
components. Technical evaluation assess whether the components function in
technical terms while keeping the objective of the integrated framework in mind
(i.e. are the components designed in such a way that they can be further integrated
to perform the foreseen analyses). This corresponds to the 'verification phase' as
defined by Sojda
(2007)
which checks that the system is complete, coherent and
logical from a modelling and programming perspective. But it is better defined by
the 'analytical validation' (checking each part of the system) as defined by Finlay
and Wilson
(1991)
. While more and more components are integrated during the
tool development, the evaluation shifts from testing individual components to
combinations of different components, evolving into the system evaluation of
whether the framework as a whole gives acceptable results for scientists which are
useful for end-users. This phase encompasses both tests of validity of solutions
of the framework as a whole (“synoptic validation” as defined by Finlay and Wilson
[1991]
) as well as analysis of solution time, extension possibilities, ease of using the
system, etc. (as identified by Mosqueira-Rey and Moret-Bonillo
2000)
.
Using Case Studies to Guide Evaluation and Development
of the Integrated Framework
Case studies played a crucial role in the evaluations of SEAMLESS-IF by providing,
real world applications of the integrated framework. Through the evaluations test
cases have become an important driver of the framework development.
In the case of SEAMLESS we opted for two types of test cases (Belhouchette
et al.
2006a, b
;
2007)
. The first deals with trade liberalization consisting of lowering
EU tariffs on imports and abolishing EU export subsidies. This case study represents
a macro (EU and global level) policy in the economic domain for which the
implications at all levels (from field to global level) and in all domains (economic,
environmental and social) are assessed. The second group of case studies challenges
