Information Technology Reference
In-Depth Information
raise likewise (
F
0
.
5
(1
,
15) = 0
.
27,
η
2
=0
.
02). In Section 5.5 we found that goal
clarity significantly raises with repetition (p=.001). We conclude that, similar to
commitment, the goal clarity is determined by learning rather than the method.
We reject
H
09
but not
H
08
.
In summary, we interpret the t.BPM method to be engaging through activa-
tion of subjects. We can not reason on the concept of identification which was
determined by other effects in this experiment. The t.BPM method also leads
to validated results through more feedback on the model. The competencies
for result validation raise partially with the method and partially with learning
through repetition.
6.1 Validity Threats
The
internal validity
was addressed by the experiment design. In particular,
we use two processes and two experimenters assigned in random order. In Sec-
tion 5.5 we assess potentially confounding variables for their influence. While
experimenters and processes did not harm the results, we found learning effects
due to the repeated measurements design on
clarity
Q
8
and
commitment
Q
5
.
We found education to be influential on the reported
clarity
Q
8
and
insights
Q
9
.
In short, oce clerks tend to report better scores while scoring worse in objective
tasks [8]. While group heterogeneity is a threat to the internal validity, it also
increases the
external validity
as both groups represent the population that we
address with our tool. This is as important as choosing domain processes rather
than artificial graphs to test with. We chose the domain processes in coordination
with the school to ensure all students are equally familiar with them. However,
wedidnotassesstowhichextendindividuals are exposed to these processes in
their companies. The
measurement instruments
were tested in one pre-study
with ten computer science students. Small adjustments were made afterwards.
To ensure quality standards for data analysis, we used two independent coders
for the video analysis and we have split each questionnaire variable into three
items, one poled negatively. Finally, we provide a longer version of this paper
including more data and the experimental material in [15].
6.2 Generalizability of Findings
We think the findings about t.BPM can be generalized from the sample group
to the general population. Besides their age (19-21years) the students represent
exactly the group we address with the t.BPM tool.
We also think that the findings will hold for other tangible modeling ap-
proaches when compared to pure talking. Some aspects have also been reported
for visual mappings of requirements such as instant feedback [20]. However, a
different tests would be needed to determine exactly the aspects that lead to
activation of participants.
6.3 Lessons Learned
If we had to start over again, we would put more effort into the reliability of our
questionnaire items, in particular
clarity
Q
8
. But we also learned that people may
