Relation Extraction : A Survey
Table 5: Comparative RE performance of various supervised approaches on 7 major relation types in ACE 2004 dataset (5-fold cross validation, LX: lexical, ST: syntactic tree, DT: dependency tree)
['Type', 'Approach', 'P', 'R', 'F']
Some of the papers in supervised RE report their performance on non-standard datasets, but some others report their performance on standard datasets such as ACE 2003 and ACE 2004. Similar to multi-class classification, the performance of supervised RE systems is evaluated in terms of precision, recall and F-measure of non-NONE classes. Even though the same dataset is used by various approaches, the actual splits/folds used in the 5-fold cross-validation might be different. Nevertheless, these figures provide a rough idea of comparative performances of these approaches. It can be observed that the kernel-based methods generally outperform the feature-based methods and among the various kernel-based methods, the methods based on syntactic tree kernel perform the best.