Latent Relation Language Models
Table 2: Perplexity values of different models on open vocabulary language modeling, lower is better. Best results are in bold. Asterisk symbols represent statistical significance according to Wilcoxon signed-rank test Dror et al. (2018) against the better model among NKLM and Vanilla LM, with p<0.05 (∗) and p<0.01 (∗∗), respectively.
['Base model', 'Dataset', 'Dev Vanilla LM', 'Dev NKLM', 'Dev LRLM', 'Test Vanilla LM', 'Test NKLM', 'Test LRLM']
We observe that for both sequence models, LRLM out-performs the baselines on all datasets (although on the one case of LSTM+WikiText-S the improvement was not statistically significant). Particularly on the two WikiText-derived datasets, our model shows significant improvements over the baselines by leveraging KGs in comparison to the vanilla LM, while NKLM has difficulty utilizing the KGs to achieve better perplexity, and in some cases results in worse perplexities than the vanilla LM.