Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation
Table 1: Evaluation metrics on CH-CHR and COV-CTR datasets comparing ASGK with other methods. C and R are shot for CIDER-D and ROUGE-L. B-n denotes that the BLEU score uses up to n-grams. Hit represents the human evaluation results.
['Dataset', 'Model', 'C', 'R', 'B-1', 'B-2', 'B-3', 'B-4', 'Hit(%)']
Automatic Evaluation. The results on both datasets indicate that ASGK outperforms all existing state-of-the-art models through its exploitation of auxiliary signals to guide the framework in knowledge pretraining and knowledge transfer procedures. The results demonstrate the robustness and superior generalization ability of ASGK. We also combine our medical graph encoder with Vision Bert Devlin et al. We adopt CIDER-D as the main metric to validate our model. On the large-scale CX-CHR dataset, ASGK significantly boosts performance compared with other baselines, it increases the CIDER score by 51.0, 35.0, 39.5, 22.1 and 22.7 respectively. However, ASGK only acheives a slightly low ROUGE-L score than the CoAttJing et al. ASGK also outperforms other baselines in COV-CTR dataset. Human Evaluation. Given 200 random images from these two datasets equally, we invited three radiologists to evaluate the corresponding outputs of our methods, CoAttJing et al. Vison-BertDevlin et al. They are encouraged to select a more accurate result from each pair. It shows that in the CX-CHR and COV-CTR datasets, radiologists thought 20%, and 27% portions of our reports are more accurate than others’ respectively, and while they thought 53%, and 22% portions of results are same. The human evaluation demonstrates that our method is capable of generating accurate and semantic-coherent reports.