Towards Understanding of Medical Randomized Controlled Trialsby Conclusion Generation
Table 4: Human evaluation results for generation quality on the rating questions.
['[BOLD] System', '[BOLD] Correctness', '[BOLD] Quality', '[BOLD] Overall']
The GPT-2 models have comparable scores in overall impression, both scoring around 3.5 out of 5. The most significant improvement of the GPT-2 generated conclusions is the text quality, with the correctness improvement to a lesser extent. The correctness of GPT-2 n=1 is slightly better than that of GPT-2 n=0 in the annotation question, yet in the rating question, GPT-2 n=0 has a higher averaged score. In sum, the human evaluation results demonstrate that our models significantly outperform the baseline pointer generator and tell that the proposed RCT conclusion generation task is not the same as typical summarization task, so deep text understanding is required for better performance.