Coherent Comment Generation for Chinese Articles with a Graph-to-Sequence Model
Table 3: Length of content, title, comment and keyword of the news for the topic of Ent (entertainment) and Sport.
['[EMPTY]', 'average word # Ent', 'average word # Sport', 'average character # Ent', 'average character # Sport']
From the number we can see that the length of news content is too large for traditional sequence-to-sequence model. We use a batch size of 32. The embedding size is set to 128. The word embeddings are shared between encoder and decoder. For all the RNN based encoders, we use bidirectional LSTM and set the hidden size to 128. For the baseline hierarchical attention model, the hidden size of the second LSTM layer is 256. We use a vocabulary size of 60,000. The sentences are truncated to 100 words. The maximum length for generating is set to 32. For multi-head attention, we use 4 heads. For RNN encoder, RNN decoder and multi-layer self-attention, we use a layer number of 2. We use a dropout rate of 0.1. The initial learning rate is set to 0.0005. For all the models, we train for 5 epochs, the learning rate is decayed to half after each epoch.