What If We Simply Swap the Two Text Fragments?A Straightforward yet Effective Way to Test the Robustness of Methods to Confounding Signals in Nature Language Inference Tasks
Table 3: Transfer test results for other downstream tasks when different percentage of training data are swapped.
['[EMPTY]', 'MR', 'CR', 'MPQA', 'SUBJ', 'TREC', 'SICK-E', 'SICK-R', 'MRPC']
As we can see, although the improvement seems marginal, the swapping operation helps improve these transfer tasks because the model is often evaluated as the best when 25%-75% of the sentences are swapped.