Information Technology Reference
In-Depth Information
Fig. 4.
Evaluation Setting: Manual (left), OntRep (right)
Study Design, Material, Participants, and Process.
We applied the standard prac-
tices of empirical software engineering research according to Freimut et al. [8] and
Wohlin et al. [22]. Fig. 4 illustrates the steps of the empirical study.
A1, B1)
Study preparation:
This step dealt with the creation/preparation of all
arti-
facts
necessary for the evaluation: 23 requirements in EBNF syntax, 8 categories as
input for the requirements categorization step, deployment of 22 seeded conflicts
(based on typical requirements conflicts found in practice at Siemens Austria). Fur-
ther, we used questionnaires to capture the individual background experience of the
participants and a feedback questionnaire to capture, whether the participants found
the approach useful and usable.
A2, A3)
Participant selection and team building
: There were 6 participants who
performed the manual categorization and conflict analysis tasks. They had similar
experience on project management (3 to 5 years) and on requirements management
but advanced general software engineering know-how. In addition, we had one expert
with deeper know-how and experience, especially in ReqM. Finally, there was one
OntRep tool user. He was well familiar with OntRep and had similar experience as
the other participants. As described below, the evaluation consisted of individual
work and team work. For the latter, the teams were assigned randomly.
A4, B2)
Introduction (Guidelines and Data Collection):
Before execution of tasks
the study organizer introduced the participants to the project and the manual require-
ments categorization and conflict detection tasks. Further, the participants were
