Abstractive Text Summarization based onLanguage Model Conditioning and Locality Modeling
Table 1: Ablation study of model with Convolutional Self-Attention on the CNN/Daily Mail dataset (kernel sizes are 11 and 3)
['Method', 'ROUGE-1', 'ROUGE-2', 'ROUGE-L']
We see that both convolutions over tokens and over attention heads improve the ROUGE scores. Standard convolution outperformed circular convolution on ROUGE-1, ROUGE-2 and ROUGE-L by 0.06, 0.13 and 0.09 percent, respectively.