Multi-Document Abstractive Summarization Using ILP based Multi-Sentence Compression
Table 1: Comparison of ROUGE scores on the DUC 2004 and 2005 datasets: Baselines, state-of-the-arts, our proposed methods and abstractive summarization system using MSC. “†” denotes the differences between [ILPSumm] and the baselines on the ROUGE scores are statistically significant for p<0.05. We limit ROUGE evaluation to 665 bytes and 250 words for the 2004 and 2005 datasets, respectively.
['[BOLD] DUC-2004', '[BOLD] DUC-2004  [BOLD] ROUGE-2', '[BOLD] DUC-2004  [BOLD] ROUGE-SU4', '[BOLD] DUC-2005', '[BOLD] DUC-2005  [BOLD] ROUGE-L', '[BOLD] DUC-2005  [BOLD] ROUGE-SU4']
Hence, we have six different systems in total. To the best of our knowledge, no publicly available abstractive summarizers have been used on the DUC dataset. In MSC, the input is a pre-defined cluster of similar sentences. Therefore, we compare our ILP based technique with MSC using the same set of input clusters obtained by our system. Hereafter, we refer to the best performing system as ILPSumm. We perform paired t-test and observe that ILPSumm shows statistical significance compared to all the baselines. The summarization method using MDImpDocsetSim measure as the most informative document and ranked by majority ordering (MO) outperforms all of the other techniques. The document that has the highest similarity to the total content captures the central idea of the documents. The clustering scheme that works best with MSC is MDImpLexRank + APO. ILPSumm also outperforms the MSC-based method, i.e., our approach can generate more informative summaries by globally maximizing content selection from multiple clusters of sentences. In summary, content selection of our proposed abstractive systems work at par with the best extractive systems.