Generating Semantically Valid Adversarial Questions for TableQA
Table 4: WikiSQL test accuracy before and after data augmentation on SQLova with BERT base encoder (SQLOVA-B). Q-Acc and A-Acc denote query and answer accuracy. AdvData-B and AdvData-L are adversarial examples generated using two different target systems, SQLova-B and the released SQLova with BERT large encoder.
['[EMPTY]', '[BOLD] AdvData-B Q-Acc', '[BOLD] AdvData-B A-Acc', '[BOLD] AdvData-L Q-Acc', '[BOLD] AdvData-L A-Acc']
Specifically, we train a SQLova system with BERT base encoder (SQLova-B) on account of efficiency. This is different from the released SQLova system with BERT large encoder. We then use SAGE to attack both SQLova-B and released system and generate two sets of adversarial examples, AdvData-B and AdvData-L. We retrain SQLova-B using the original training data augmented with 30k and the full (56k) adversarial examples from AdvData-B and AdvData-L, respectively. Although AdvData-L is not targeted for SQLova-B, it can still boost the performance of SQLova-B.