Extractive Summarization as Text Matching
Table 5: Results on test sets of WikiHow, PubMed and Multi-News. MatchSum beats the state-of-the-art BERT model with Ngram Blocking on all different domain datasets.
['[BOLD] Model', '[BOLD] WikiHow  [BOLD] R-1', '[BOLD] WikiHow  [BOLD] R-2', '[BOLD] WikiHow  [BOLD] R-L', '[BOLD] PubMed  [BOLD] R-1', '[BOLD] PubMed  [BOLD] R-2', '[BOLD] PubMed  [BOLD] R-L', '[BOLD] Multi-News  [BOLD] R-1', '[BOLD] Multi-News  [BOLD] R-2', '[BOLD] Multi-News  [BOLD] R-L']
Ngram Blocking has little effect on WikiHow and Multi-News, and it causes a large performance drop on PubMed. We think the reason is that Ngram Blocking cannot really understand the semantics of sentences or summaries, just restricts the presence of entities with many words to only once, which is obviously not suitable for the scientific domain where entities may often appear multiple times.