SynsetRank: Degree-adjusted Random Walk for Relation Identification
TABLE I: Area under the ROC curve (AUC) for 13 relations of the FB15k-237 dataset.
['Relation', 'Frequency', 'Moro', 'PageRank', 'PageRank (common)', 'SynRank', 'SynRank (common)']
In order to avoid data sparsity issues, we determined the twenty relations with the highest number of mentions in the training partition, and removed seven from these which were redundant with respect to the other relations or which were semantically lightweight from the point of view of textual mentions (see ‘Frequency’ denotes the baseline method where the synsets are ranked based on the original frequency \bfp(0). We clearly observe that the plain PageRank tends to perform worse than Moro et al. ’s baseline method, while our proposed degree-adjusted SynsetRank shows drastically better performance than all others. For PageRank and SynsetRank, the optimal parameters (maximizing the AUC on the validation partition) are grid-searched over α=0.0,0.2,…,1.0, β=0.0,0.2,…,1.0, and t=1,…,5 for each relation. For PageRank (common) and SynsetRank (common), the common optimal parameters (maximizing the average AUC over all 13 relations) are used. Likewise, the third and the fourth removed relations are close to the eighth evaluated relation, and the fifth and the sixth removed relations are close to the seventh evaluated relation. Here, ‘semantically close’ means that the synsets relevant for the respective relations are very likely to be the same, and we can expect that the result on each removed relation is similar to its closest evaluated relation. The seventh removed relation is unlikely to be mentioned at all in the text, so is not interesting for relation extraction. Accordingly, we removed those seven relations to reduce the hand-labeling work (∼8 person-hours per relation).