Justifying Diagnosis Decisions by Deep Neural Networks
Table 4: Results for image captioning experiments where B1 = BLEU1, B2 = BLEU2, B3 = BLEU3, B4 = BLEU4, R = Rouge, M = Meteor, C = CIDEr, σ = standard deviation for the CIDEr metric over three runs. The results of the convolutional captioning architecture of (Aneja et al., 2018) with and without pretraining are compared to our method where the best model is selected based on either the label accuracy or the caption score on the validation set.
['Model', 'B1', 'B2', 'B3', 'B4', 'R', 'M', 'C', '[ITALIC] σ']
We use a convolutional captioning model with attention mechanism that was recently shown to outperform the traditional captioning setup that combines a convolutional image feature network and LSTM captioning model While this model uses a pretrained ImageNet classifier network that is subsequently finetuned during training, we also compare it to a setup where the classifier network is trained from scratch as the ImageNet features would not necessarily be useful for X-Ray images. While the BLEU and ROUGE scores focus on co-occurrence statistics based on n-grams, METEOR aligns the captions to reference sentences and calculates sentence-level similarity scores. CIDEr is an evaluation score that is specifically aimed at image captioning tasks with multiple descriptive sentences per image.