[
Table 2: Automatic Evaluation Scores. The table is divided into natural language metrics and clinical finding accuracy scores. BLEU-n counts up n-gram for evaluation, and accuracy is the averaged macro accuracy across all clinical findings. Major class always predicts negative findings.
['[EMPTY]', '[BOLD] Model', '[BOLD] Natural Language CIDEr', '[BOLD] Natural Language ROUGE', '[BOLD] Natural Language BLEU-1', '[BOLD] Natural Language BLEU-2', '[BOLD] Natural Language BLEU-3', '[BOLD] Natural Language BLEU-4', '[BOLD] Clinical Accuracy']
Ours (NLG), that solely optimizes CIDEr score, achieves superior performance in terms of natural language metrics, but its clinical meaningfulness is not significantly above the major class in which we predict all patients to be disease-free. This phenomenon is common among all other models that do not consider the clinical alignment between the ground truth and the generated reports. On the other hand, in our full model, if we consider both natural language and clinical coherence, we can achieve the highest clinical disease annotation accuracy while still retaining decently high NLG metrics.