Automatic Evaluation of Ontologies - Natural Language Processing and Text Mining - page 195

ity, unity, etc.) that can be used to better understand the nature of various kinds of
semantic relationships that commonly appear in ontologies, and to discover possible
problematic decisions in the structure of an ontology. For example, a property is
said to be essential to an entity if it necessarily holds for that entity. A property
that is essential for all entities having this property is called rigid (e.g., “being a
person”: there is no entity that could be a person but isn't; everything that is a per-
son is necessarily always a person); a property that cannot be essential to an entity
is called anti-rigid (e.g., “being a student”: any entity that is a student could also
not be a student). A class defined by a rigid property cannot be the subclass of a
class defined by an anti-rigid property. This observation allows us to conclude, if we
see an ontology in which “person” is a subclass of “student,” that this relationship
is wrong. Various other kinds of misuse of the is-a relationship can also be detected
in a similar way (for example, is-a is sometimes used to express meta-level charac-
teristics of some class, or is used instead of is-a-part-of, or is used to indicate that a
term may have multiple meanings). A downside of this approach is that it requires
manual intervention by a trained human expert familiar with the above-mentioned
notions such as rigidity; at the very least, the expert should annotate the concepts
of the ontology with appropriate metadata tags, whereupon checks for certain kinds
of errors can be made automatically. As pointed out, e.g., in [12], applications where
evaluation of this sort is truly important (and justifies the costs) are probably rela-
tively rare. However, Volker et al. [28] recently proposed an approach to aid in the
automatic assignment of these metadata tags.
Maedche and Staab [16] propose several measures for comparing the relational
aspects of two ontologies. If one of the ontologies is a gold standard, these measures
can also be used for ontology evaluation. Although this is in a way a drawback of
this method, an important positive aspect is that once the gold standard is defined,
comparison of two ontologies can proceed entirely automatically. The semantic co-
topy of a term c in a given hierarchy is the set of all its super- and sub-concepts.
Given two hierarchies H 1 , H 2 , a term t might represent some concept c 1 in H 1 and
a concept c 2 in H 2 . One can then compute the set of terms which represent concepts
from the cotopy of c 1 in H 1 , and the set of terms representing concepts from the
cotopy of c 2 in H 2 ; the overlap of these two sets can be used as a measure of how
similar a role the term t has in the two hierarchies H 1 and H 2 . An average of this
may then be computed over all the terms occurring in the two hierarchies; this is a
measure of similarity between H 1 and H 2 .
Similar ideas can also be used to compare other relations besides is-a. Let R 1
be a binary relation in the first ontology, with a domain d ( R 1 ) and a range r ( R 1 ).
Analogously, let R 2 be a binary relation in the second ontology. We can consider the
relations to be similar if d ( R 1 ) is similar to d ( R 2 ) and r ( R 1 ) is similar to r ( R 2 ). Since
d ( R 1 ) and d ( R 2 ) are simply two sets of concepts, they can be compared similarly
as in the preceding paragraph: determine the set of terms that occur as names of
any concept of d ( R 1 ) or any of its hypernyms; in analogous way, determine the
set of terms for d ( R 2 ); then compute the overlap of these two sets. The overlap
between ranges r ( R 1 ) and r ( R 2 ) can be computed in an analogous way. If there
are several such pairs of relations, the similarity can be computed for each pair and
then averaged to obtain an indicator of relational-level similarity between the two
ontologies as a whole.