Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA
Table 3: Ablation for the Control unit and Bridge-entity supervision, reported as EM scores after training on the regular or adversarial Add4Docs-Rand data, and evaluation on regular dev set and Add4Docs-Rand adv-dev set. Note that 1-hop Base is same as 2-hop without both control unit and bridge-entity supervision.
['Train Eval', 'Regular', 'Regular Adv', 'Adv Regular', 'Adv']
S5SS0SSS0Px4 Control Unit Ablation We also conduct an ablation study on the 2-hop model by removing the control unit. The results validate our intuition that the control unit can improve the model’s multi-hop reasoning ability and robustness against adversarial documents.