Barack’s Wife Hillary:Using Knowledge Graphs for Fact-Aware Language Modeling
Table 3: Perplexity Results on Linked WikiText-2. Results for models marked with * are obtained using importance sampling.
['[EMPTY]', '[BOLD] PPL', '[BOLD] UPP']
We observe that the KGLM attains substantially lower perplexity than the other entity-based language models (44.1 vs. 76.1/85.4), providing strong evidence that leveraging knowledge graphs is crucial for accurate language modeling. Furthermore, KGLM significantly outperforms all models in unknown penalized perplexity, demonstrating its ability to generate rare tokens.