Evaluating the Stability and Credibility of Ontology Matching Methods

Presented at: 8th Extended Semantic Web Conference (ESWC2011)

by Xing Niu, Haofen Wang, Gang Wu, Guilin Qi, Yong Yu

Ontology matching is one of the key research topics in the field of Semantic Web. In the last few years, many matching methods have been proposed to generate matches between different ontologies either automatically or semi-automatically. To select appropriate ones, users need some measures to judge whether a method can achieve the similar compliance even on one dataset without reference matches and whether such a method is reliable w.r.t. its output result along with the confidence. However, widely-used traditional measures like precision and recall fail to provide sufficient hints on them. In this paper, we design two novel evaluation measures to evaluate stability of matching methods and one measure to evaluate credibility of matching confidence values, which help to answer the above two questions. Additionally, we carry out comparison among several carefully selected methods systematically using our new measures. Besides, we report some interesting findings such as identifying potential defects of our subjects.

Keywords: Credibility, Evaluation, Experiments, Ontology Matching, Stability

Resource URI on the dog food server: http://data.semanticweb.org/conference/eswc/2011/paper/ontologies/7

Explore this resource elsewhere: