Information Technology Reference
In-Depth Information
manual approach?
In order to address the research question we derived the following
variables (according to [8]) to consider for evaluation:
•
The
number of requirements
determines the effort necessary for categorization
and conflict analysis.
•
Number of requirement categories
used to categorize the requirements. Further,
the
total number of true requirements conflicts
existing in a list of requirements,
which can be identified by various approaches for conflict detection. This is a
baseline measurement for the effectiveness of an approach, i.e., a perfect ap-
proach would find 100% of the true requirements conflicts.
•
Approach for categorization/conflict analysis
: e.g., for automation approaches
the formal structure of requirements is an important factor. As described above,
we use the EBNF template for specifying requirements. Using plain text or other
formats probably affects the correctness and completeness of identified conflicts.
Dependent variables that we want to study by the evaluation are:
•
Number of conflicts identified
. This number consists of two measures: recall
(number of correctly identified conflicts), for measuring the effectiveness of an
approach, and false positives (number of wrongly identified conflicts).
•
True conflicts that have not been identified
(false negatives): subtracting the
number of correctly identified conflicts
from the
total number of true require-
ments conflicts
tells us about the recall of conflict identification.
•
Plausibility of requirements classification
: regarding categorization two kinds of
error can occur: (1) requirements have been assigned to a wrong category, and (2)
requirements have not been assigned to a category they actually belong to. In or-
der to measure these parameters, we take the manual categorization results of a
requirements engineering expert as reference. In addition, we count the
number of
requirements in each category
.
Besides these parameters we also record the effort for requirements categorization and
conflict analysis. This includes
preparation effort
(e.g., creating the ontology that is
used for categorization and conflict analysis),
categorization effort
, and
conflict
analysis effort
. The case study is described in detail in the following section.
4 Case Study Description
The following subsections describe the characteristics of the pilot study design.
Study Subject.
The case study project “Technoweb 2.0” is an IT development project
with the goal to design and implement a web application that serves as a platform for
communication and networking between technology experts within Siemens. This
platform builds on the Java technology
Liferay
, where
portlets
act as components of
the web application. The project is performed in an agile way using the software de-
velopment process “SCRUM” and the configuration & project management platform
Trac
. In Trac all requirements, tasks and bugs are stored as tickets.
