Latent Relation Language Models
Table 5: Model and training hyperparameters that are common across the models. Slash-delimited values represent different hyperparameters used in WikiFacts, WikiText-S, WikiText-F, respectively.
['Common hyperparameters Learning rate decay rate', 'Common hyperparameters 0.9']
A2SS0SSS0Px2 Hyperparameters While we use the same Transformer-XL hyperparameters across datasets, we apply different sets of LSTM hyperparameters on WikiFacts, WikiText-S, and WikiText-F for better performance. We take pre-trained KG embeddings from OpenKE Han et al.