PAWS: Paraphrase Adversaries from Word Scrambling
Table 4: Paraphrase judgments on example pairs generated by back translation on Wikipedia sentences.
['Total # back translation pairs', '26,897']
The swapping strategy generally produces non-paraphrase examples—67% for QQP and 88% for Wikipedia. Because (a) the label imbalance is less pronounced for QQP and (b) NMT models perform poorly on Quora questions due to domain mismatch, we only apply the back translation strategy to Wikipedia pairs. Doing so creates 26,897 candidate example pairs after filtering. As before, each pair is rated by five annotators on the paraphrase identification task.