Information Technology Reference
In-Depth Information
4.3 Results
Result of our experiment is summarized in Table 2. We record the retrieved
model patterns of both approaches per the process model pairs.
With comparison to the gold standard mentioned in Section 2, we calculated
the precision rate of the approaches under study, as shown in Table 3. Precision
rate is a measure commonly used to evaluate the quality of search and mining
techniques [11].
Tabl e 2.
Experimental Result
Approach A
Approach B
)or
(
{A, B, E, F, O, J, K} , {D, E}
)or
(
{P,Q,R} , {M,N}
)
(
{S, T} , {D, E}
(
{P,Q,R, S, T, A,F, I, J,K,M, D,E}
,
(a) and (b)
{D, E, F,G,H,M, N, I, J,K, L}
)
(c) and (b)
(
{E,H} , {F, G}
)
or
(
{E,H} , {D, E}
)or
(
{T,U} , {D, F}
)
or
(
{A, D} , {D, F}
)
(
{B,C,D} , {D,E, F,G}
)
(
{A, B, C, D, E, F, G, H, I} , {D, E}
)or
(
{A, B, C, D, E, F, G, H, I} , {G, H}
)or
(
(
{A, B, C, D, E, F, G, H, I}
,
{F,G,H, I, J,K, L,M}
(d) and (b)
)
{A, B, C, D, E, F, G, H, I} , {L, M}
)
Tabl e 3.
Precision of Approach A and Approach B
Approach (a) and (b) (c) and (b) (d) and (b)
Only consider semantic correspondences 0.10344828 0.1369863 0.14285714
Add topological consideration
0.33333333 0.27027027 0.41666667
As we can observe that, given a target model (Process (b) in the experiment),
the retrieved results from Approach A are more scattered than the ones from Ap-
proach B. Also we see Approach A allows more irrelevant patterns as compared
to Approach B. The main reason for that is in Approach A the topological infor-
mation is omitted, which actually counts a lot when comparing process models.
Moreover, due to the strict adjacent relation, we have little merging work done
on the correspondence sets. That explains the scattered distribution in the result
of Approach A. Clearly, in the real-life application, we need to extend such limit
for a more meaningful retrieved result.
In view of the two observations in the experiment on Approach A, we pro-
pose Approach B, with loose adjacent relation and topological consideration.
The result shows the validity of this optimization, with significantly increased
precision rate, as well as the more reasonable retrieved result in comparison with
Approach A, as shown in Table 2 and 3.
