Context-aware Natural Language Generation with Recurrent Neural Networks
Table 2: Results of language modeling measured by perplexity (P: product, S: sentiment).
['Domain', 'RNN', 'C2S(P)', 'C2S(S)', 'C2S(P+S)', 'gC2S(P)', 'gC2S(S)', 'gC2S(P+S)']
We start with the task of language modeling. This shows that contextual information are indeed helpful for natural language understanding and prediction. Second, the product context seems to be more informative than the sentiment context for language modeling, no matter with the C2S or gC2S model. This may be that there are many words in the reviews that are relevant to the product information. Comparing the C2S and gC2S model, the gC2S model consistently outperforms the C2S model no matter which contexts are used. As explained previously, this is because in the C2S model, the context information may not be able to affect the words that are far away from the beginning of the sequences while the gC2S effectively addresses this through adding the direct dependency between the contexts and the words in the sequences.