Information Technology Reference
In-Depth Information
expert was also the best with the manual approach: He needed 45 min. for conflict
analysis, whereas the other participants needed 97 min. on average. In addition the
group phase took 37 min., resulting in an additional group effort 111 person minutes.
5.3 Threats to Validity
We addressed threats internal validity [10] of the study by two measures: a) intensive
reviews of the study concept and materials, and b) a test run of the study conducted by
a test person in order to make sure that the guidelines, explanations, and task descrip-
tions are understandable for the participants and to estimate the required effort/time
frame. Regarding external validity [30], we performed this initial case study in a pro-
fessional context at a software development company. The participants had medium
requirements management know-how and advanced software engineering know-how.
In addition, we had a requirements engineering expert as experimental “control
group”. Nevertheless, the small number of participants might limit the generalization
of results. Therefore, we suggest replicating the study in a larger context.
Further, the requirements in this case study were formulated using the EBNF syn-
tax, which is a major condition for OntRep to analyze the requirements. We did
not yet analyze the quality of results with a set of requirements, which is not or only
partially formulated in EBNF. Further studies are needed to evaluate this.
6 Discussion and Conclusion
Software and systems engineering projects are complex due to the increasing number
and complexity of requirements, and the project participants with different domain
backgrounds and terminologies. To keep the overview on requirements, project man-
agers conduct requirements categorization, conflict analysis, and tracing. However,
the manual conduct of these tasks takes significant effort and is error-prone.
In this paper we proposed semantic technology as foundation for automating the
requirements management tasks and introduced the automated ontology-based report-
ing approach OntRep based on a project ontology and a reasoning mechanism. We
used requirements formulated in EBNF as input to the proposed OntRep approach,
which supports automated requirements
categorization
and requirements
consistency
checking.
We evaluated the effectiveness and effort the OntRep approach based on a
real-world industrial case study with 6 project managers in 2 teams. The study fo-
cused on requirements categorization and requirements conflict analysis. During the
evaluation the study participants a) categorized the requirements of the case study
project into a set of categories and b) inspected the given project requirements to
identify conflicts between requirements. In addition a requirements expert and an
OntRep user performed the same tasks to enable comparing the quality of results and
the effort for all activities.
The case study results suggest that OntRep can be an attractive alternative for
requirements categorization in typical software development projects, because it pro-
vides slightly lower effectiveness with similar effort compared to manual approaches,
but much more scalable. OntRep's performance can be increased by adding addi-
tional, synonyms or hyponyms to the ontology (which has to be done manually at the
