Information Technology Reference
In-Depth Information
is calculated as TP/(TP + FP). Precision is commonly paired with Recall, which is
calculated in the same way as Sensitivity. Precision and Recall can be combined
into a single statistic that is the harmonic mean, called the F1-score or F-measure:
(Pr × Sn)/(Pr + Sn).
Regardless of what approach might be used for evaluation, it is important to be
fully cognizant of the limitations of each approach. That is, evaluation is relative to
a given gold standard or expertise used in an ad hoc assessment. Nonetheless, it is
essential to perform evaluation of knowledge discovery systems, especially in the
context of bibliome mining. One valid criticism of gold standard based evaluation is
that it may not accurately assess the value of a bibliome mining system to assess
new knowledge. Furthermore, one might consider that evaluation against a gold
standard only validates that a system is able to identify what is already known.
Thus, in the context of bibliome mining, ad hoc evaluations are more commonplace.
This is because the evaluation is based on a true comparison between machine
inferred knowledge and human expertise. There are certainly limitations to this
approach (e.g . , having consistent evaluations across experts); however, such limita-
tions can be addressed in part by metrics such as the aforementioned Cohen's or
Fleiss' Kappa statistics. It is important to continuously evaluate biblioming systems
and provide benchmark evaluations based on statistically meaningful samples.
5.3
Bibliome Mining to Support a Learning
Healthcare System
Within biomedicine, bibliome mining is an area of ongoing research. Many bibli-
ome mining systems have been developed with the ultimate goal of identifying
putative hypotheses that can be used to inform clinical decisions. The nuances and
complications with how data and thus knowledge are embedded within biomedical
literature continue to challenge the research community. However, there is great
potential for identifying potentially useful knowledge that may be actionable using
the bibliome mining systems that have been developed to date. The relevance of
knowledge that may be embedded within biomedical literature may indeed be the
underpinning support for the identifi cation of innovations and their subsequent eval-
uations in the context of a learning healthcare system. As described in Chap. 1 , the
promise of a learning healthcare system is that it is one that knowledge associated
with healthcare are seamlessly fed-forward (to identify new knowledge) and fed-
back (to quantify the effect of using identifi ed knowledge). Bibliome mining may
very well be an essential process that enables this paradigm shift from the current,
static environment of knowledge sharing. For example, considering the previously
described discovery of the possible treatment of Raynaud's syndrome using cod
liver oil from study of literature (by Swanson), it is essential to consider how to dis-
seminate such knowledge into clinical practice and also evaluate the effect at the
population level.
Search WWH ::




Custom Search