Environmental Engineering Reference
In-Depth Information
System Evaluation of Quantitative Tools
A system evaluation of quantitative tools in an integrated framework will focus on
validation of the outcomes of the linked set of components included in the frame-
work. One of the big issues here is the problem of chain of errors and uncertainty
(Gabbert et al.
2009)
. Another issue whether problems with the compatibility of
different components (models or databases) has been solved through the ontology.
System Evaluation of the User Interface
The key question of the user interface is whether it provides access to results that
are useful and transparent for end-users. Added to this may be requirements on the
way results are displayed, which could vary by type of end-user. These requirements
can to some extent be determined beforehand (and are also part of the technical
evaluation using the test cases). Use of the operational framework by end-users will
no doubt lead to additional requirements, just as in the evaluation of the integrated
assessment procedure evaluation criteria were defined during testing.
Some More Lessons to Organize the Evaluation
of an Integrated Framework
The type of testing done for SEAMLESS-IF is summarized in Table
10.1
.
The experiences gained can be translated in several key points which can be of use for
other projects aiming at evaluating integrated assessment tools: use of prototypes,
Table 10.1
Summary of evaluation levels used for the different types of components
Conceptual evaluation
Technical evaluation
System evaluation
Procedure (e.g.
SEAMLESS-IF
procedure for IA)
− Consistency with
overall objectives
− Operational
feasibility
Organisational
feasibility
− Relevance for users
− Reliability
(quality of outcome)
Quantitative tools
(e.g. APES)
Consistency with overall
objectives and with
major characteristics
of the system it
simulates (a crop)
− Reliability
(outputs quantitative
analysis)
− Uncertainty
analysis
− Programming
problems
− Input/output
analysis
− Operational
feasibility
User interfaces
(e.g. GUI)
Consistency with overall
objectives
Operational feasibility
