Justifying Diagnosis Decisions by Deep Neural Networks
Table 5: Average score given by human experts for each method for 4 different questions. For all questions, a higher score is better. The corresponding p-value is also given for a two-tailed equal variance t-test.
['Question', 'Saliency map', 'Our', 'p-value']
All p-values are calculated with two-tailed equal variance t-tests. Note that the proposed justification method significantly outperforms in the task of explaining the diagnosis that was put forth as measured by question 1. When analyzing the answers of each clinician separately, the results hold. For the 30 X-Rays that are analyzed by all three experts, we calculate the inter-annotator agreement in the form of Fleiss’ kappa for the first three questions. The outcomes are 0.33, 0.42 and 0.55 respectively, demonstrating that the evaluation is balanced and consistent across all human experts.