Optimizing the Factual Correctness of a Summary:A Study of Summarizing Radiology Reports
Table 2: Main results on the two datasets. R-1, R-2, R-L represent the ROUGE scores. PG Baseline represents our baseline augmented pointer-generator; RLR, RLC and RLR+C represent RL training with the ROUGE reward alone, with the factual correctness reward alone and with both. All the ROUGE scores have a 95% confidence interval of at most ±0.6. F1 scores for extractive models were not evaluated for the reason discussed in Section 5.3.
['System', '[BOLD] Stanford R-1', '[BOLD] Stanford R-2', '[BOLD] Stanford R-L', '[BOLD] Stanford Factual F1', '[BOLD] RIH R-1', '[BOLD] RIH R-2', '[BOLD] RIH R-L', '[BOLD] RIH Factual F1']
We first notice that while the neural extractive model, BanditSum, outperforms the non-neural extractive method on ROUGE scores, our PG baseline model substantially outperforms both of them, suggesting that on both datasets abstractive summarization is necessary to generate summaries comparable to human-written ones.