Generating Abstractive Summaries from Meeting Transcripts
Table 6: Content selection evaluation. We compute ROUGE-2 (R-2) and ROUGE-SU4 (R-SU4) scores by comparing the system generated summaries and the human-written summaries for all the meetings in the test set.
['[BOLD] Method', '[BOLD] R-2', '[BOLD] R-SU4']
We use ROUGE-2 (R-2) and ROUGE-SU4 (R-SU4) recall scores to compare all approaches. Furthermore, the ROUGE scores of the MSC model also significantly outperforms the extractive model, indicating that MSC results in more informative summaries. To evaluate the impact of anaphora resolution, we run our abstractive summarization model without performing the pre-processing step of pronoun resolution. ROUGE-2 score obtained by the abstractive model with anaphora resolution (0.048) is significantly better than the model without anaphora resolution (0.036). This indicates that anaphora resolution significantly contributes to content selection. Due to pronoun resolution, there are more chances of fusing information from various utterances within a topic segment. The extractive model obtains the lowest ROUGE scores as we restrict comparison to the first 300 words. In contrast, the abstractive methods (our methods and MSC) can effectively integrate the information from multiple utterances within the first 300 words.