Paraphrase Detection on Noisy Subtitles in Six Languages
Table 2: Classification accuracies on the Opusparcus test sets for models trained on 1 million positive sentence pairs. AP (all paraphrases) is the majority baseline, which is the accuracy obtained if all sentence pairs in the test data are labeled as paraphrases. Consistent improvement is obtained by the WA model without segmentation (WA-M: “WA without Morfessor”) and further by the WA model with segmentation. Whether the GRAN model outperforms WA is hard to tell from these figures, but this is further analyzed in Section 4.2.
['[EMPTY]', 'AP', 'WA-M', 'WA', 'GRAN']
The differences in performance between the WA models with segmentation (called just WA) and without segmentation (called WA-M) clearly indicate that this is a necessary preprocessing step when working on languages with complex morphology. The effect of segmentation for the GRAN model (not shown) is similar, with the exception of English also improving by a few points instead of worsening. Based on these results we will use Morfessor as a preprocessing step in all of the remaining experiments.