Relation Discovery with Out-of-Relation Knowledge Base as Supervision
Table 4: Comparison of prediction results based on encoder using NYT122, NYT71, and NYT27 datasets with different KB regularization strategies.
['[BOLD] Model', '[BOLD] NYT122 F1', '[BOLD] NYT122 F1', '[BOLD] NYT122 NMI', '[BOLD] NYT122 NMI', '[BOLD] NYT71 F1', '[BOLD] NYT71 F1', '[BOLD] NYT71 NMI', '[BOLD] NYT71 NMI', '[BOLD] NYT27 F1', '[BOLD] NYT27 F1', '[BOLD] NYT27 NMI', '[BOLD] NYT27 NMI']
S4SS3SSS0Px2 Comparison on Different Datasets. We also compare our algorithm on the three datasets with different baseline settings. In order to evaluate our model rigorously, besides the original DVAE model, we compare two additional augmented baseline models with the same hyper-parameter setting: DVAE with TransE embeddings appended to encoder input features (DVAE+E) and DVAE with decoder entity vectors replaced by pre-trained KB embeddings (DVAE+D). For our method, we report RegDVAE with the best setting where we use Euclidean distance based constraints to regularize the encoder. Moreover, we report a setting with fixed embeddings in the decoder as the ones obtained from TransE (RegDVAE+D). This also makes sense since even though the TransE embeddings are not trained with the observation of the same relations as the text corpus, the embeddings already contain much semantic information about entities. Then by fixing the embeddings of entities in the decoder, we can significantly reduce the number of parameters that need to be trained. As we can see that, RegDVAE+D can outperform the original DVAE by 8∼23 points on F1. DVAE+D is also good but may fail when there are a lot of out-of-sample entities in the training corpus.