Information Technology Reference
In-Depth Information
4 Research Method
4.1 Design and Measures
To test our hypotheses, we conducted an experiment in which the participants
had to answer questions on a BPM. Model understandability (in terms of cor-
rectly answered questions) and perceived subjective diculty were measured
at each of the four levels
order
,
concurrency
,
repetition
and
exclusiveness
of the
factor “type of comprehension question”. To manipulate the main factor we con-
structed comprehension questions targeting the four different relations between
activities.
Comprehension Questions.
When selecting the questions, we took into con-
sideration the work of Melcher et al. [18]. However, in comparison to [18] we for-
mulated questions consistently, so that participants always had to consider two
model elements (two activities) and their relationship for answering a question.
Additionally we tried to use every-day-language in the questions. We used two
different wordings to ask for the four relations between activities. To demon-
strate the type of questions we refer to two activities with alphabetic names,
although A and B were replaced with activity labels in the test material:
-
Concurrency:
•
“A and B can be executed at the same point of time.”
“A and B can be executed in parallel.”
-
Exclusiveness:
•
•
“In one process instance, A as well as B can be executed.”
“The activities A and B are mutually exclusive.”
-
Order:
•
•
“If A as well as B are executed in a process instance, then A has to be
finalized, before B can start.”
“If A as well as B are executed in a process instance, then A is executed
before B.”
-
Repetition:
•
•
“A can be executed more often than B.”
•
“In each process instance A is executed exactly as often as B.”
We took care that the wording in the questions is understandable, and we ran
a pre-test in order to make sure that the participants understood the questions
[24]. The comprehension questions, to which participants had to give a response
of “right”, “wrong” or “I don't know”, were selected so that each activity was
addressed approximately once in each diagram. The response option “I don't
know” was included to lower guessing probability.
Questionnaire Construction.
For each model in the questionnaire we posed
the same eight types of comprehension questions. Despite the use of the same
wording, it is obvious that there is a large number of possibilities how to ask these
questions, because any two activities can be targeted with the same question. We
identified two basic variations: 1) the statement given in the question is correct
