Generating Representative Headlines for News Stories
Table 3. Performance comparison of different methods. H stands for training with human-curated labels. L/S/D denotes Language model pre-training, Single-document pre-training and Distant supervision pre-training respectively.
['[BOLD] Method', 'R1-P', 'R1-R', '[BOLD] R1-F', 'R2-P', 'R2-R', '[BOLD] R2-F', 'RL-P', 'RL-R', '[BOLD] RL-F', '[BOLD] Len-C', '[BOLD] Len-W']
It proposes a decoder-only module with memory-compressed attention for the abstractive stage. Concat first concatenates body texts in all articles of a story, and then uses the single-document encoder-decoder building block of our model to generate the headline. This way, every attention layer in Transformer is able to access tokens in the entire story. To avoid losing word position information after concatenation, the positional encoding is reset at the first token for each article, so the model can still identify important leading sentences of each article. SinABS It uses a referee attention module to integrate encoding outputs from different articles as the representation for the article set. For fair comparison, we replace the original LSTM encoder with the transformer architecture, with the same parameter size as our model. SinABS (enhanced): The original model only makes use of knowledge from single articles. We further enhance it with distant supervision ( Generally, abstractive methods outperform extractive methods, even though the latter have access to already summarized information from existing article titles. Among abstractive methods, the carefully designed concatenation model can achieve comparable performance with the existing state-of-the-art. When enhanced with the distantly supervised training data (H+LSD), the SinABS model can have stronger performance. Our model, when trained with the same resources, consistently outperforms baseline methods.