Pre-training via Paraphrasing
Table 3: Unsupervised Sentence Retrieval results on BUCC. MARGE outperforms other unsupervised models.
['[EMPTY]', 'en-de', 'zh-en']
On Tatoeba (see Appendix), there is significant variation across languages, but overall MARGE performs comparably to XLM-R and significantly better than other pre-trained models. Better results have been achieved on both tasks using labeled bitext for training (