Databases Reference
In-Depth Information
We can then define precision as the percentage of instances of
C
, which are also in-
stances of
A
and recall as percentage of instances of
A
, which are also instances of
C
. This is visualised in Figure 29. F-Measure is defined as harmonic mean of pre-
cision and recall. For learning super classes, we use
F
3
measure by default, which
gives recall a higher weight than precision.
2.
A-Measure:
We denote the arithmetic mean of precision and recall as A-Measure.
Super class learning is achieved by assigning a higher weight to recall. Using the
arithmetic mean of precision and recall is uncommon in Machine Learning, since
it results in too optimistic estimates. However, we found that it is useful in super
class learning, where
F
n
is often too pessimistic even for higher
n
.
3.
Generalised F-Measure:
Generalised F-Measure has been published in [36] and
extends the idea of F-measure by taking the three valued nature of classification
in OWL
DLs into account: An individual can either belong to a class, the negation
of a class or none of both cases can be proven. This di
/
ers from common binary
classification tasks and, therefore, appropriate measures have been introduced (see
[36] for details). Adaption for super class learning can be done in a similar fashion
as for F-Measure itself.
4.
Jaccard Distance:
Since
R
(
A
)and
R
(
C
) are sets, we can use the well-known Jaccard
coe
ff
cient to measure the similarity between both sets.
Fig. 29.
Visualisation of di
ff
erent accuracy measurement approaches.
K
is the knowledge base,
A
the class to describe and
C
a class expression to be tested. Left side: Standard supervised
approach based on using positive (instances of
A
) and negative (remaining instances) examples.
Here, the accuracy of
C
depends on the number of individuals in the knowledge base. Right side:
Evaluation based on two criteria: recall (
Which fraction of R
(
A
)
is in R
(
C
)
?
) and precision (
Which
fraction of R
(
C
)
is in R
(
A
)
?
).
We argue that those four measures are more appropriate than predictive accuracy when
applying standard learning algorithms to the ontology engineering use case. Table 4
provides some example calculations, which allow the reader to compare the di
ff
erent
heuristics.
E
cient Heuristic Computation.
Several optimisations for computing the heuristics are
described in [82]. In particular, adapted approximate reasoning and stochastic approxi-
mations are discussed. Those improvements have shown to lead to order of magnitude
gains in e
ciency for many ontologies. We refrain from describing those methods in
this chapter.
