SAM: Semantic Attribute Modulation for Language Modeling and Style Variation
Table 4: Corpus-level perplexity with Title-Attribute on (a) BBCNews and (b) IMDB
['Attributes Source', 'Method', 'BBCNews', 'IMDB', 'TTNews', 'XLyrics']
We show the word prediction perplexity results in Table. The RNN-based models, with the title embedding, has better perplexity results. Moreover, SAM-Title is better than RNN-state because the added title information would disappear after several nonlinear gating functions. The attention-based title attribute performs better than the one without attention. This is because the attention mechanism provides the different importance weights for the title words. We use the semantic attribute attention to conjoin the two attributes and the suffix ‘Au’ means that this method incorporates the author categorical attribute and maintains the method notations used in the previous part. We show the word prediction perplexity results of several attribute combinations in Table. For the TTNews and XLyrics datasets, we can see that incorporating both title and author attributes are better than the single one.