Optimizing the Factual Correctness of a Summary:A Study of Summarizing Radiology Reports
Table 5: Perplexity scores as evaluated by the trained radiology impression LM on the test set human references and model predictions.
['System', 'Stanford pplx.', 'RIH pplx.']
We find that while extractive models can achieve non-trivial overlap with references, their perplexity scores tend to be much higher than humans. We conjecture that this is because radiologists are trained to write the summaries with more compressed language than when they are writing the findings, therefore sentences directly extracted from the findings tend to be more verbose than needed.