Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization
TABLE V: Ablation test to investigate the effect of different constraints.
['[EMPTY]', 'SDA EM', 'SDA F1', 'SEA EM', 'SEA F1']
We conducted ablation test to investigate the effect of each constraint. MemReader was chosen as the base model and it was evaluated on adversarial examples to discriminate between different constraints. On one hand, adversarial examples violating the former two constraints are observed more frequently than those violating the predicate constraint in our data. On the other hand, determining SEA and SDA examples via entity/lexical constraints is more straightforward and precise than predicate constraint. Second, models with more constraints perform better than those with less constraints: three constraints (Full) > two constraints (−∗) > single constraint ( Only *) > no constraint (Adv-Training).