Adversarial NLI: A New Benchmark for Natural Language Understanding
Table 8: Percentage of development set sentences with tags in several datasets: AdvNLI, SNLI, MuliNLI and FEVER. ‘%c’ refers to percentage in contexts, and‘%h’ refers to percentage in hypotheses. Bolded values label linguistic phenomena that have higher incidence in adversarially created hypotheses than in hypotheses from other NLI datasets, and italicized values have roughly the same (within 5%) incidence.
['[EMPTY]', '[BOLD] Other Datasets  [BOLD] SNLI', '[BOLD] Other Datasets  [BOLD] SNLI', '[BOLD] Other Datasets  [BOLD] MNLI [ITALIC] m', '[BOLD] Other Datasets  [BOLD] MNLI [ITALIC] m', '[BOLD] Other Datasets  [BOLD] MNLI [ITALIC] mm', '[BOLD] Other Datasets  [BOLD] MNLI [ITALIC] mm', '[BOLD] Other Datasets  [BOLD] F', '[BOLD] ANLI  [BOLD] A1', '[BOLD] ANLI  [BOLD] A1', '[BOLD] ANLI  [BOLD] A2', '[BOLD] ANLI  [BOLD] A2', '[BOLD] ANLI  [BOLD] A3', '[BOLD] ANLI  [BOLD] A3']
We compare the incidence of linguistic phenomena in ANLI with extant popular NLI datasets to get an idea of what our dataset contains. We observe that FEVER and SNLI datasets generally contain many fewer hard linguistic phenomena than MultiNLI and ANLI