Neural Headline Generation with Sentence-wise Optimization
Table 4: Effect of using different restrictions of output vocabulary on English development set.
['[EMPTY]', '[ITALIC] R1 [ITALIC] R', '[ITALIC] R2 [ITALIC] R', '[ITALIC] RLR']
To reduce the computation complexity when training NHG model, a possible approach is to restrict the size of vocabulary for generated headlines. There are three typical methods to deal with the size. The first one is to restrict the output words of headline within the input sentence, denoted as Input-only. The second one is to construct an extended vocabulary that includes similar words with those appear in the input sentence, denoted as Extended-input. The third one is to use full vocabulary, denoted as Full-vocab. We observe that the “Input-only” and ”Extened-input” achieve comparable performance while using hundreds of times less vocabulary. It indicates that it is feasible to utilize these tricks to train NHG model much more efficiently.