Does William Shakespeare REALLY Write Hamlet? Knowledge Representation Learning with Confidence
Table 2: Evaluation results on entity prediction
['Datasets Metric', 'FB15K-N1 Mean Rank', 'FB15K-N1 Mean Rank', 'FB15K-N1 Hits@10(%)', 'FB15K-N1 Hits@10(%)', 'FB15K-N2 Mean Rank', 'FB15K-N2 Mean Rank', 'FB15K-N2 Hits@10(%)', 'FB15K-N2 Hits@10(%)', 'FB15K-N3 Mean Rank', 'FB15K-N3 Mean Rank', 'FB15K-N3 Hits@10(%)', 'FB15K-N3 Hits@10(%)']
we demonstrate the results of entity prediction with different noise rates, from which we can observe that: (1) all confidence-aware KRL models consistently and significantly outperform baseline on all noisy datasets with all evaluation metrics. It confirms the quality of learned knowledge representations, for they could not only detect noises in knowledge graphs, but also perform well in knowledge graph completion. (2) Comparing with evaluation results between different datasets, we find that the improvements introduced by our confidence-aware methods become more significant as the noise rate in KGs goes higher. It indicates that noises are harmful to entity prediction, and on the other hand reaffirms that considering triple confidence in knowledge representation learning is essential. (3) It seems that the global path confidence has few contributions on entity prediction. It may be partially caused by the uncertainty and incompleteness in path information due to possible error propagation and limited path selection. In parameter analysis, we find that a higher weight of global path confidence within a reasonable range will improve the performances of entity prediction while harming those of KG noise detection. Taking longer relation paths into consideration with prior knowledge or patterns to assure high path qualities will partially solve this problem. (4) For a more comprehensive comparison, we evaluate TransE on the original FB15K dataset (without any noises) with the same parameter settings, evaluation protocols and test set. The result of “Hits@10 (Filter)” is 64.7% and “Mean Rank (Filter)” is 134, which can be viewed as the upper bound of our models. Compared to these results, the improvements of CKRL seem to be good as well.