Violin: A Large-Scale Dataset for Video-and-Language Inference
Table 5: Accuracy (%) on test set containing negative statements collected via different strategies. Image and BERT features are used in this experiment.
['Method', 'Annotated', 'Adversarial matching']
Human-Written vs. Adversarially-Sampled Negatives For comparison, we create a new statement set by replacing the adversarially-sampled negative statements with original human-written negative statements. Performance on the sampled negatives is higher than that on human-written ones. Our interpretation is that human-written content has higher propensity for intent understanding and in-depth reasoning, which makes the statements more challenging to the model.