Environmental Engineering Reference
In-Depth Information
The evaluations of the SEAMLESS-IF procedure were performed with 11
different policy experts (or groups of experts) and eight modellers (or pairs
of modellers). Policy experts who were consulted are representative of different
types of institutions (government, resource management institution, agriculture
advice services, local public institutions) acting in different political fields
such as water management, resource management, agriculture and at different
levels from local to regional and national. Modellers were SEAMLESS researchers
of different disciplines from biophysical (agro-ecology) to social sciences
(e.g. economics, geography). They had different levels of knowledge of the
framework and its components. This diversity of policy experts and integrative
modellers ensured that the procedure (and accordingly the relevance of the framework
specifications) was evaluated in a large range of situations so that the outcome
of the evaluation exercises were not narrowed to specific issues and working
conditions.
Conceptual Evaluation of Quantitative Tools
Integrated assessment tools combine quantitative tools in a single framework. In the
case of SEAMLESS three groups can be distinguished: databases (and ontology),
numerical models and indicators. These components are of course interrelated,
within the modelling chains including the database and the procedure to compute
indicators for sustainability assessment. The requirements on each component
depend on its place in the overall framework. The conceptual evaluation first
determined the expected contribution of each component to the overall objective of
the integrated framework (i.e. does it deliver the information required by the whole
framework). Then each component was assessed individually (i.e. does it provide
the information properly). The two case studies were essential in these evaluations
by providing concrete requirements for the whole system from which requirements
on components could be derived. This illustrates also the need to carefully choose
case studies representative of the target range of studies, because by guiding the
conceptual evaluation they drive further development of the components.
Different types of quantitative tools require different evaluation procedures.
Databases are based on the development of ontology that should encompass
the different data, procedures and concepts mobilized in SEAMLESS-IF. Using
object-oriented databases and specific UML diagrams helped at analysing the
links between these different entities and the conceptual functioning between
databases and other quantitative tools (models and indicators). Conceptual
evaluation of numerical models is easier because these models describe, in their
concepts and equations, a part of the reality the users have in mind for the real
systems under study. This principle has been used to communicate with users and
detect missing information (Dieste et al.
2003)
. Such conceptual evaluation was
mainly based on flowchart and expert appraisal. Conceptual evaluation of indicators
was based on a five-points grid: presence of the indicator (“Is it requested by
