Tangible Media in Process Modeling – A Controlled Experiment - Advanced Information Systems Engineering - page 287

Time Slicing( H 1 , H 2 ): The duration of the experimental task is sliced exclu-
sively to belong to one of five categories. The use of t.BPM ( Use tBPM )suchas
labeling and positioning the shapes without talking, Talk tBPM/int is the time
people talk about the process, UseTalk tBPM is talking and using t.BPM (to
avoid overlap between Use tBPM and Talk tBPM ). We define a code for the time
spent silent ( Silence tBPM/int ) when people do not talk and do not handle t.BPM.
Finally, Rest tBPM/int captures remaining time such as interactions with the in-
terviewer. The same coding scheme is used for both experimental tasks. However,
Use and UseTalk do not apply for interviews as there is no t.BPM to use.
Corrections and Reviews( H 6 , H 7 ): Both are coded as distinct events. We
code Corrections tBPM/int if the context of an already explained process part is
explicitly changed. In t.BPM sessions this involves re-labeling or repositioning
that impacts the process model meaning. In interviews explicit revisions of pre-
viously stated information is considered a correction. The Reviews tBPM/int are
coded if subjects decide to recapitulate their process. This must involve talking
about the process as we cannot account possibly silent reviews. This scheme is
the same for both experimental tasks.
Using this coding scheme we operationalize the video hypotheses in the
following way:
•

H 1 : Subjects talk more in t.BPM sessions than in interviews,
i.e. Talk tBPM + UseTalk tBPM >Talk int .
H 01 : Subjects don't talk more in t.BPM sessions,
i.e. Talk tBPM + UseTalk tBPM >Talk int .

• H 2 : Subjects are more silent in t.BPM sessions than in interviews,

i.e. Silence t.BPM >Silence int
H 02 : Subjects are not more silent in t.BPM sessions,
i.e. Silence t.BPM >Silence int

• H 6 : Subjects make more reviews in t.BPM sessions than in interviews,

i.e. Reviews t.BPM >Reviews int
H 06 : Subjects don't make more reviews in t.BPM sessions,
i.e. Reviews t.BPM

>Reviews int

• H 7 : Subjects make more corrections in t.BPM sessions than in interviews,

i.e. Corrections tBPM > Corrections int .
H 07 : Subjects don't make more corrections in t.BPM sessions,
i.e. Corrections tBPM

> Corrections int .

3.6 Variables
The independent variable in this experiment setup is the method used for process
elicitation. Subjects do either a structured interview or the same structured
interview in the presence of t.BPM, the tangible modeling toolkit. The dependent
variables are formed from the data collected during and immediately after a
session. We use a notational convention for the data sets collected: intention V/Qx .
As an example, talking V 1 describes the set of talking times as measured with the
video analysis for hypothesis 1. Likewise, fun Q 3 is the set of all ratings collected
with the fun related questionnaire items for hypothesis 3.