Improving Context Aware Language Models
Table 5: Comparison of perplexities per subreddit
['[BOLD] Subreddit', '[BOLD] Base. PPL', '[BOLD] Adapt. PPL', 'Δ [BOLD] PPL', '[BOLD] Description']
It can be seen that the relative gain from adaptation is largest when the topic of the subreddit is more narrowly focused. The biggest gains were achieved for subreddits dedicated to specific sports, tv shows, or video games. Whereas, the gains were smallest for subreddits like Videos or Funny whose content tends to be more diverse. The knowledge that a sentence came from a pro-wrestling subreddit effectively provides more information about the text than the analogous piece of knowledge for the Pics or Videos subreddit. This would seem to indicate that further gains could be possible if additional contextual information could be provided. An alternative explanation, that subreddits with less sentences in the training data receive more benefit from adaptation, is not supported by the data.