Adversarial Examples for Evaluating Reading Comprehension Systems
Table 5: Transferability of adversarial examples across models. Each row measures performance on adversarial examples generated to target one particular model; each column evaluates one (possibly different) model on these examples.
['Targeted Model', 'Model under Evaluation ML', 'Model under Evaluation ML', 'Model under Evaluation BiDAF', 'Model under Evaluation BiDAF']
AddSent adversarial examples transfer between models quite effectively; in particular, they are harder than AddOneSent examples, which implies that examples that fool one model are more likely to fool other models. The AddAny adversarial examples exhibited more limited transferability between models. For both AddSent and AddAny, examples transferred slightly better between single and ensemble versions of the same model.