Adversarial Training for Commonsense Inference
Table 3: Development and test results of CosmosQA, MCScript 2.0 and MCTACO. The best results are in bold. Note that RoBERTaLARGE, SMART and ALICE models use RoBERTaLARGE as the text encoder, and for a fair comparison, all these results are produced by ourselves. On the test results, note that CosmosQA and MCTACO are scored by using the official evaluation server (https://leaderboard.allenai.org/). * denotes unpublished work and scores were obtained from the evaluation server on April 16, 2020. Acccs denotes the accuracy on commonsense based questions and Accood denotes the accuracy on questions that are not commonsense based, i.e. out-of-domain questions.
['Model', '[BOLD] CosmosQA Acc', '[BOLD] MCScript2.0 Acc', '[BOLD] MCScript2.0 Acccs', '[BOLD] MCScript2.0 Accood', '[BOLD] MCTACO EM', '[BOLD] MCTACO F1']
BERT + unit normalization Zhou et al. The authors further add unit normalization to temporal expressions in candidate answers and fine-tune on the MC-TACO dataset. RoBERTaLARGE is our re-implementation of the large RoBERTa model by Liu et al. K-ADAPTER Wang et al. further enhances RoBERTa word representations with multiple knowledge sources, such as factual knowledge obtained through Wikipedia and Wikidata and linguistic knowledge obtained through dependency parsing web texts. SMART Jiang et al. SMART uses the model prediction, f(x;θ), for estimating the perturbation δ. This model recently obtained state-of-the-art results on a bunch of NLP tasks on the GLUE benchmark Wang et al. We also compare ALICE with a baseline that uses only the label y for estimating the perturbation δ (called model ADV hereafter) Madry et al. Overall, we observed that adversarial methods, i.e. ADV, SMART and ALICE, were able to achieve competitive results over the baselines, without using any additional knowledge source, and without using any additional dataset other than the target task datasets. These results suggest that adversarial training lead to a more robust model and help generalize better on unseen data.