Information Technology Reference
In-Depth Information
The average effort for manual categorization was around 15 min per person. The
group work took ca. 12 min in addition, resulting in an additional group effort of 36
person minutes. With OntRep the following preparations were necessary to enable the
automated categorization: conversion of requirements into EBNF form (30 min.),
preparation of ontology classes and user-defined synonyms (14 min.). After this, the
run time for categorization was ca. 2 minutes. If the requirements exist in EBNF form,
which is the case for some larger projects at Siemens Austria, the effort is similar to
the manual average effort of manual categorization, but much more scalable.
5.2 Requirements Conflict Analysis
We analyzed conflicts of the three types described above, because conflicts of this
type can be modeled in OntRep by means of facts and rules. The OntRep results for
these conflicts are complete: all 22 conflicts of the defined conflict classes in the
given data were identified, because OntRep works reliably, when the following pre-
requisite are met: requirements exist in EBNF as input via CSV, modeling of glos-
sary terms in ontology (10 min.), modeling facts, constraints and rules in the ontology
(46 min.). Therefore, the total OntRep preparation time is 100 min. for the given case.
The overall report generation took 4 minutes.
Table 2.
Results of conflict detection capability analysis
# correctly identified
conflicts
(avg./
std.dev.)
Avg. % of true
conflicts found
(avg./
std.dev.)
# conflicts
found
(avg./
std.dev.)
False positives in
% (avg./
std.dev.) of #
conflicts found
Individuals
7.0/3.9
31.8/18
17.0/6.8
58.8/22.0
Groups
10.5/0.7
47.7/3.0
21.5/0.7
51.2/4.9
Expert 15.0 68.2 17.0 11.8
OntRep
22.0 100.0 22.0 0.0
In comparison, the manual conflict analysis approach resulted in a lower complete-
ness (see Table 2): the individual participants identified only 31.8% of existing
conflicts on average. The harmonization of results within the groups brought an im-
provement to 47.7%, which means that approximately 3 additional conflicts have
been identified by merging of the individual results into one group result. Also the
number of false positives was slightly reduced by 1. The correctness of the manual
approach was also lower than with the OntRep approach: 58.8% of identified conflicts
were false positives. This percentage could only slightly be reduced by the group
harmonization. i.e., ca. 1-2 false positives were been eliminated during team work.
In addition to comparing the individual and group results with OntRep results, but
also had one expert performing the conflict analysis. Compared to the other participants
and the team results, he provided the best results, i.e., the highest number of correctly
identified traces, and the lowest percentage of false positives. Regarding effort, the
