Database Reference
In-Depth Information
Fifteen Dutch workfl ow designers working for management consulting companies,
one large bank, and one utilities provider, were asked to cooperate in this survey. Fourteen
of them actually responded before the deadline. The average number of workfl ow design
projects they participated in ranged from two to 25, with an average of 10.
For the analysis of the results we used statistical methods for
rater agreement
, as often
used in medical settings to compare experts' opinions on the same data. For more background
on rater agreement statistics, see Uebersax (1992, 2001). We computed the Pearson cor-
relation between the respondents' average score and the heuristic outcome. In Table 3, we
have represented the outcomes of the web survey for each of the respondents (R1…R14)
for each of the dilemmas (D1…D10). The average score for each dilemma is given in the
column labeled 'R_avg'; the heuristic's outcome column with 'H'.
The correlation between the average respondent score and the heuristic outcome ap-
proximately turned out to be 0.810. This is a signifi cant result, assuming a two-tailed 99%
confi dence interval.
In addition to this fi rst analysis, we examined the relation between the average respon-
dent score and each individual correspondent (
Corr R_avg
), as well as the relation between
the heuristic outcome and each individual respondent (
Corr H
). The respective correlation
coeffi cients are shown in Table 4.
In addition to this analysis, we interviewed respondents R2, R5, R7, R11 and R13 the
day following their survey. Of particular interest was the opinion of respondent R11. He
explained that in general he is in favor of splitting up activities in a workfl ow design in the
smallest possible parts. His consideration is that at run-time execution of a workfl ow process,
activities may be dynamically combined to be allocated to workers if this looks like a good
idea under the circumstances. Taking a look at the results in Table 3 for respondent R11,
Figure 3: Screenshot of the web-survey
