[
Table 2: Comparison of disease classification performance using AUROC. * results are from wang2018tienet.
['Disease', 'Ate', 'Cardio', 'Effusion', 'Infil', 'Mass', 'Nodule', 'Pneum', 'Pneumox', '[BOLD] Average']
Evaluation of Disease Classification. Although ROUGE and CIDEr scores are effective in evaluating the consistency of an automatic report to a manual report, none of them, however, are designed for assessing the correctness of medical report annotation in terms of common thoracic diseases. The latter is another key output of a useful image interpretation system. For example, the automatically generated sentence: “no focal airspace consolidation, pleural effusion or pneumothorax” is considered as similar to the manually written sentence: “persistent pneumothorax with small amount of pleural effusion” using both ROUGE and CIDEr scores despite the completely opposite annotations. Therefore, we assess the accuracy in medical report annotation by comparing with TieNet wang2018tienet in disease classification using Area Under the ROC (AUROC) as the metric.