Databases Reference
In-Depth Information
100 %
100 %
Completeness
Minimality
Structurality
Scbema-
Protimity
Completeness
Minimality
Structurality
Scbema-
Proximity
80 %
80 %
60 %
60 %
40 %
40 %
20 %
20 %
0%
0%
COMA +++
Currency Dataset
SF
COMA +++
Betting Dataset
SF
100 %
100 %
Completeness
Minimality
Structurality
Schema-
Protimity
Completeness
Minimality
Structurality
Schema-
Proximity
80 %
80 %
60 %
60 %
40 %
40 %
20 %
20 %
0%
0%
COMA +++
SF
COMA +++
SF
SMS Dataset
University Courses Dataset
Fig. 9.11
Experimental results for the evaluation of the target schema quality
notions may not have yet matured in the minds of researchers and of the commercial
developers and users, and to avoid confusions, we have first attempted to provide a
complete description of the architectural components, tasks, and goals of matching
and mapping tools. Then, we have motivated the importance of evaluation methods
and benchmarks for researchers, developers, businesses, and users.
Schema matching is a topic that has been extensively studied. There is already a
long list of research prototypes and tools. Since the matching task involves seman-
tics, evaluating the correctness of the output of a matching tool is a task requiring
human intervention. The major issue in all these matching cases is deciding what is
the correct answer, i.e., the intended matches. This is a challenging task since, due to
the semantic heterogeneity, different perspectives may give different answers. Eval-
uation techniques for matching tasks have focused on the development of metrics
that will allow a common evaluation base and effective communication of the eval-
uation results. We have provided a description of these metrics and have highlighted
features and limitations.
Schema mapping seems to be a problem for which there is still some confusion
as to what constitutes a mapping tool, what is its input, in what form, and what is its
output. Different research prototypes and commercial tools have followed different
approaches, something that makes their direct comparison and evaluation difficult.
We have attempted to provide a definition of what a mapping tool is and the param-
eters one should consider when evaluating such tools. We have highlighted the lack
of evaluation standards and have provided a complete picture of what an evalua-
tion standard (or benchmark) should contain, alongside existing efforts toward the
creation of such a standard.
