In Silico Hypothesis Discovery - Translational Informatics: Realizing the Promise of Knowledge-Driven Healthcare - page 139

Agreement Matrix 1 Agreement Matrix 2 Agreement Matrix 3

0 1 4 6 0 0 1 3
2 1 4 5 6 0 1 1
0 0 0 2 3 4 5 6
1 1 1 2 3 4 5 4
1 0 0 1 1 1 2 2
0 0 4 2 1 0 0 0

0 1 4 6 0 0 1 3
2 1 4 5 6 0 1 1
0 0 0 2 3 4 5 6

0 1 4 6 0 0 1 3
2 1 4 5 6 0 1 1
0 0 0 2 3 4 5 6

1 1 1 2 3 4 5 4
1 0 0 1 1 1 2 2
0 0 4 2 1 0 0 0

1 1 1 2 3 4 5 4
1 0 0 1 1 1 2 2
0 0 4 2 1 0 0 0

Concatenated Agreement Matrix

0 1 4 6 0 0 1 3 0 1 4 6 0 0 1 3 0 1 4 6 0 0 1 3
2 1 4 5 6 0 1 1 2 1 4 5 6 0 1 1 2 1 4 5 6 0 1 1
0 0 0 2 3 4 5 6 0 0 0 2 3 4 5 6 0 0 0 2 3 4 5 6
1 1 1 2 3 4 5 4 1 1 1 2 3 4 5 4 1 1 1 2 3 4 5 4
1 0 0 1 1 1 2 2 1 0 0 1 1 1 2 2 1 0 0 1 1 1 2 2
0 0 4 2 1 0 0 0 0 0 4 2 1 0 0 0 0 0 4 2 1 0 0 0

System
Complexity
(Consensus)

Random Matrices
(Probability Distribution)

Contribution of
System

Pattern
Recognition

Information
Content

0 1 1 0 0 3 3 1
0 1 1 0 0 3 3 1

0 1 1 0 0 2 3 1
0 0 1 2 3 4 0 0
6 6 1 2 3 0 0 1
2 3 4 0 0 1 2 0
1 1 2 0 0 4 0 1
1 1 0 0 0 1 1 1

LESS

Fig. 8.11 Overview of information theoretic evaluation method for determining the degree of
multi-source or expert agreement within a knowledge collection or system

Design Rule Checking (DRC) algorithms [ 43 ]. All of these algorithms use some
combination of the number of and distance between interrelated vertices within the
graph as the basis for determining cohesion. Most cohesive graphs generally pos-
sess more interrelated vertices with relatively short edges between them. However,
it is important to note that a precise defi nition of what constitutes “cohesion” in a
graph is not necessarily universally agreed upon. Due to this lack of agreement,
class cohesion algorithms tend to utilize different measures for cohesion. The appli-
cability of these metrics varies depending on the specifi c evaluation context. As
a result, the selection of an appropriate cohesion measure is highly dependent on
the specifi c nature of the data set and application scenario being evaluated. Further
details concerning the theoretical basis and application of graph theory-based cohe-
sion measures can be found in the review provided by Zhou et al. [ 43 ].

8.4.5 Logical Methods

The application of logic-based verifi cation and validation techniques for the out-
put of intelligent agents focuses on the detection of axiomatic consistency. These
techniques require the extraction of logical axioms from the knowledge collection
that has informed such in silico hypothesis discovery operations. Once axioms have
been extracted, they are then applied within the targeted domain in order to evaluate