Point-less: More Abstractive Summarization with Pointer-Generator Networks
Table 5. Duplicated N-grams within summaries for multi-head models where cov stands for models that have been trained with coverage.
['Model extensions', '[EMPTY]', '1-grams', '2-grams', '3-grams', '4-grams', 'Sentences']
To understand the relationship between the coverage loss and our proposed losses we trained all models without the coverage mechanism. Further, it can be seen that the number of novel N-grams decreases significantly. This matches our expectations for N-grams ¿ 1 which reflect the order of words. However, we can also observe a decrease in novel 1-grams which suggests that the coverage mechanism favours extractive summarization which stands in contrast to our goal towards a more abstractive model. The losses thus interfere with coverage. Coverage reduces repetition at the expense of more extractive summaries. Penalizing the pointing mechanism introduces new N-grams but also increases the problem of overgeneration.