Towards Understanding of Medical Randomized Controlled Trialsby Conclusion Generation
Table 2: ROUGE scores of the PGNet baseline models and the GPT-2 fine-tuned models on the development set. The GPT-2 res were trained with the “results” section only. Addition of n>1 hint words did not show further gains in ROUGE scores.
['[BOLD] System', '[BOLD] ROUGE-1', '[BOLD] ROUGE-2', '[BOLD] ROUGE-L']
Note that the hint words are not considered in score calculation and the output of all models are lower-cased. The GPT-2 fine-tuned model significantly outperforms the pointer-generator (PGNet) baseline on all ROUGE scores, where the best performing model is GPT-2 with hint word n=1, demonstrating the effectiveness of generating good conclusion in our model. However, more hint words do not bring additional gain in ROUGE scores, probably because more constraints hinder the GPT-2 model to explore potentially good conclusions. Moreover, the ablation result shows the significant drop in all ROUGE scores, indicating the importance of including the “background” and “objective” sections in the input for better content understanding.