Extractive Summarization as Text Matching
Table 1: Datasets overview. SDS represents single-document summarization and MDS represents multi-document summarization. The data in Doc. and Sum. indicates the average length of document and summary in the test set respectively. # Ext denotes the number of sentences should extract in different datasets.
['[BOLD] Datasets', '[BOLD] Source', '[BOLD] Type', '[BOLD] # Pairs Train', '[BOLD] # Pairs Valid', '[BOLD] # Pairs Test', '[BOLD] # Tokens Doc.', '[BOLD] # Tokens Sum.', '[BOLD] # Ext']
Since the appearance of the pearl-summary will bring challenges to sentence-level extractors, we attempt to investigate the proportion of pearl-summary in different datasets on six benchmark datasets. In the case of short summaries (Reddit and XSum), the perfect identification of pearl-summaries does not lead to much improvement. Similarly, multiple sentences in a long summary (PubMed and Multi-News) already have a large degree of semantic overlap, making the improvement of the summary-level method relatively small. But for a medium-length summary (CNN/DM and WikiHow, about 60 words), the summary-level learning process is rewarding. As the gold summaries get longer, the upper bound of summary-level approaches becomes more difficult for our model to reach. MatchSum can achieve 0.64 ψ(D) on XSum (23.3 words summary), however, ψ(D) is less than 0.2 in PubMed and Multi-News whose summary length exceeds 200. From another perspective, when the summary length are similar, our model performs better on datasets with more pearl-summaries. We use an approximate method here: