Scoring Sentence Singletons and Pairs for Abstractive Summarization
Table 2: Instance selection results; evaluated for primary, secondary, and all ground-truth sentences. Our BERT-SingPairMix method achieves strong performance owing to its capability of building effective representations for both singletons and pairs.
['[EMPTY]', '[BOLD] System', '[BOLD] Primary  [BOLD] P', '[BOLD] Primary  [BOLD] R', '[BOLD] Primary  [BOLD] F', '[BOLD] Secondary  [BOLD] P', '[BOLD] Secondary  [BOLD] R', '[BOLD] Secondary  [BOLD] F', '[BOLD] All  [BOLD] P', '[BOLD] All  [BOLD] R', '[BOLD] All  [BOLD] F']
Extraction Results we present instance selection results for the CNN/DM, XSum, and DUC-04 datasets. Our method builds representations for instances using either BERT or VSM To ensure a thorough comparison, we experiment with selecting a mixed set of singletons and pairs (“SingPairMix”) as well as selecting singletons only (“SingOnly”). On the CNN/DM and XSum datasets, we observe that selecting a mixed set of singletons and pairs based on BERT representations (BERT+SingPairMix) demonstrates the most competitive results. It outperforms a number of strong baselines when evaluated on a full set of ground-truth sentences. The method also performs superiorly on identifying secondary sentences. For example, it increases recall scores for identifying secondary sentences from 33.8% to 69.8% (CNN/DM) and from 16.7% to 65.3% (XSum). Our method is able to achieve strong performance on instance selection owing to BERT’s capability of building effective representations for both singletons and pairs. It learns to identify salient source content based on token and position embeddings and it encodes sentential semantic compatibility using the pretraining task of predicting the next sentence; both are valuable additions to summary instance selection.