Information Technology Reference
In-Depth Information
4.2 Materials
Questionnaire Parts.
We used a pencil-and-paper questionnaire including
three different sections in the experiment. The first section comprised items
to obtain information about participants' demographic data, academic quali-
fications and modeling experience. Participants were asked about the number
of years they had worked in the IT sector and the extent to which they had
previously been involved with modeling in the context of education and work.
After the first section, the questionnaire included a tutorial on process modeling,
which covered all aspects the participants would need to know to perform the
comprehension tasks. The third section included four different models with eight
corresponding comprehension tasks per model. The amount of models used was
determined by the selection of the comprehension questions during the experi-
ment, as we wanted to ask 32 different instances of comprehension questions. To
avoid order effects due to decreasing motivation or concentration of participants,
we used two different scramblings. Models as well as comprehension questions
were presented in different order, respectively.
Model Domain.
The four models were selected from different domains such
that we could expect that they are understandable for an average student with
no special domain knowledge.
Model Language.
Because it has been shown that the graphic style of a model-
ing language can influence the understandability of the model [25], we presented
BPM modeled using different graphic styles. The models were modeled in differ-
ent modeling directions and with three different routing symbol designs (UML
Activity Diagrams, BPMN and Event-Driven Process Chains). These variations
were included for allowing to generalize findings beyond specific layout and de-
sign restrictions. Additionally, they served as an experimental control to prevent
a possible bias due to choosing only one modeling direction and routing symbols
from a specific modeling language for all diagrams.
Model Layout.
We took into account that a change in the graphical layout of a
BPM can influence its comprehensibility [9]. For this reason, we took care that the
graphical layout of the models did not impose additional challenges to the reader.
Model Size.
Each of the four models used contained 21 activities. The model
size was held constant for all models, because this variable is likely to have an
influence on cognitive load of answering understandability questions.
4.3 Participants
A total of 199 business students participated in this study (125 males, 74 fe-
males), aged 23.5 years on average. Of all respondents, 36% were undergraduate
students, 60% were master's level students and 4% had already completed their
master's degrees. 67% had received training in modeling at university with 1.6
credit hours on average. About half of participants were familiar with Event-
Driven Process Chains (60%) and UML Activity Diagrams (50%). 27% had
work experience in the IT industry and 10% had already worked with BPMs.
