A procedural model for ontological analyses - Information Systems Foundations: Constructing and Criticising - page 160

2. Step 2: The researchers involved in Step 1 of the methodology, meet to discuss and
defend their interpretations of the representation modelling analysis. This meeting
should lead to an agreed second draft version of the analysis that incorporates ele-
ments of each of the researchers' first draft analyses. The overlap in the selection
of the grammatical constructs and in the actual ontological analysis can be quantified
by various figures that are used in content analysis and other more qualitative re-
search.
3. Step 3: The second draft version of the analysis for each of the interoperability
candidate standards is used as a basis for defence and discussion in a meeting in-
volving the entire research team. The outcome of this meeting forms the final ana-
lysis of the grammar in question.
Just such a method was employed in a project that sought to apply the BWW represent-
ation model analysis to a number of the leading potential Web Services standards:
ebXML, BPML, BPEL4WS and WSCI. The project team was composed of four researchers
and the standards were analysed in the order: ebXML à BPML à BPEL4WS à WSCI. Two
researchers were involved in Steps 1 and 2 of the method (the individual analysis of a
standard followed by a meeting of the two researchers in order to obtain an agreed
mapping). This was followed by a meeting of the entire team in order to discuss the
mapping and arrive at the final analysis. The process was performed for each of the four
standards. Table 13.1 shows the recorded agreement statistics at the second step of the
applied method while Table 13.2 shows the recorded agreement statistics at the third
step of the method.
Table 13.1. Summary of Step 2 mapping agreement between both researchers
Mapping conference Web Service Language Construct Mapping

agreed upon by both
researchers

Total number of
specification constructs
identified

ebXML 43 51 84%

BPML 36 46 78%

BPEL4WS 30 47 63%

WSCI
Table 13.2. Summary of Step 3 mapping agreement

39 49 79%

Web Service Language Construct Mapping

agreed upon by the team

Total number of
specification constructs
identified

Mapping conference

ebXML 49 51 96%

BPML 41 46 89%

BPEL4WS 42 47 89%

WSCI
The adoption of such a method can be seen to have greatly improved the objectiveness
of the carried-out analyses.

46 49 94%

Output
The three main shortcomings related to the outcome of an ontological analysis have been
characterised as the lack of adequate result representation, lack of result classification
and the lack of relevance.
The meta models, which have been used as input for the ontological analyses, are an
appropriate medium to visualise the outcomes of the entire analysis process. In our work
on the analysis of ARIS, we derived a meta model of the BWW model that highlighted
all constructs of the ontology that did not have a corresponding construct in the grammar
under analysis. That is, we visualised incompleteness in the model using simple colour