NCLS: Neural Cross-Lingual Summarization
Table 5: Experimental results on different versions of datasets. Filter refers to the version of dataset for which we employ RTT strategy to filter. Non-Filter denotes the version of the dataset obtained by simply forward translation without filtering process including back translation. Pseudo-Filter is the dataset randomly sampled from Non-Filter version and is of the same size as Filter version. BT refers to back translation in RTT. For En2Zh task, we train the TNCLS (w-c). For Zh2En task, we train the TNCLS (sw-sw).
['DataVersion', 'BT?', 'En2ZhSum', 'En2ZhSum*', 'Zh2EnSum', 'Zh2EnSum*']
Why Back Translation? To show the influence of filtering the corpus by back translation during the RTT process, we use three kinds of datasets to train our TNCLS models and compare their performance. They are: (a) the CLS dataset obtained by simply employing forward translation on MS dataset (Non-Filter) ; (b) the CLS dataset obtained by a complete RTT process (Filter); (c) the dataset obtained by sampled from Non-Filter dataset to keep the same size as the Filter dataset (Pseudo-Filter). The models trained on Filter dataset significantly outperform the models trained on Pseudo-Filter dataset on both En2Zh and Zh2En tasks, which indicates that the back translation can effectively filter dirty samples and improve the overall quality of corpora, thus boosting the performance of NCLS. In En2Zh task, the model trained on Non-Filter dataset performs best. The reasons are two-fold: (1) the quality of machine translation for English news is reliable; (2) the scale of Non-Filter dataset is almost twice that of the two others so that after the amount of data reaches a certain level, it can make up for the noises caused by the translation error in the corpus. In Zh2En task, the performance of the model trained on Non-Filter dataset is not as good as that on Filter. It can be attributed to the fact that current MT is not very ideal in the translation of texts on social media so that the dataset constructed by only using forward translation contains too many noises. Therefore, when the quality of machine translation is not that ideal, backward translation is especially important during the process of constructing corpus.