Information Technology Reference
In-Depth Information
6 ES
0.38%
Distribution of
Annotation
Accuracy
4 ES
0.25%
47 ES
2.97%
80 ES
5.05%
168 ES
10.61%
0.4
0.5
0.6
0.7
0.8
0.9
1
122 ES
7.71%
1156 ES
73.03%
Accuracy
Categories
(40%-100%)
Fig. 4.
Result Categories of the Annotation Accuracy Experiment
briefed on the workings of the service annotation approach. He then manually
performed the annotation based on the technical names of these 10% sample
Enterprise Services. In a final step, the expert's manual annotations have been
analyzed and compared to the automatically generated annotations by one of
our developers. As a result, we confirmed that approximately 94% of automated
annotations match the manual annotations in terms of number of annotations
and annotated concepts. We further investigated the 6% range of annotation
mismatches, which we separated into two categories: governance violation and
missing record. The former category describes Enterprise Services that, for what-
ever reason, did not fully comply with the SOA Governance design methodology.
The latter denotes that the specific concept is missing in the service knowledge
base and has been wrongly annotated as a different concept. While mismatches
resulting from governance violations are out of our control, mismatches stem-
ming from missing records can be fixed by extending the knowledge base. Either
way, we believe that mismatches of less than 6% can be considered acceptable.
6 Related Work
Before we examine related work on proposed annotation techniques, we first
outline some fundamental approaches that have been proposed to add addi-
tional knowledge (metadata) to the technical description of services, e.g WSDL.
For instance, linguistic approaches using clustering techniques - as done in the
Woogle search engine [8] - aim to derive meaningful concepts from the techni-
cal description of Web Services using WSDL. Although it effectively improves
the search, the approach lacks certainty about generated concepts making an
