Guiding Extractive Summarization with Question-Answering Rewards
Table 5: Amazon mechanical turk experiments. Human abstracts were the goldstandard summaries, Pointer+Cov. were summaries generated by See et al. See et al. (2017). Our systems tested were the supervised extractor, and our full model (NER).
['[BOLD] Summary', '[BOLD] Time', '[BOLD] Accuracy', '[BOLD] Inform.']
Excluding the use of human abstracts, all systems resulted in similar performance times. However we observe a large margin in QA accuracy in our full system compared to the abstractive and our supervised approach. Although participants rated the informativeness of the summaries to be the same our systems yielded a higher performance. This strongly indicates that having a system which makes using of document comprehension has a tangible effect when applied towards a real-world task.