Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering
Table 5: Ablation study, removing different components of ROCC. The scores are reported on the ARC test set and MultiRC dev set. R⋆ denotes the best approach that relies just on the R score. The hyper parameter k in R⋆, was tuned on the development partition of the respective dataset.
['#', 'Ablations', 'ARC', 'MultiRC EM0', 'MultiRC Justification']
Row 0 reports the score from the full AutoROCC model. In all the cases, we found small drops in both performance and justification scores across both the datasets, with the removal of either C(A) or C(Q) having the largest impact.