Decomposable Neural Paraphrase Generation
Table 4: In-domain performance of paraphrase generation.
['Models', 'WikiAnswers→ Quora BLEU', 'WikiAnswers→ Quora iBLEU', 'WikiAnswers→ Quora ROUGE-1', 'WikiAnswers→ Quora ROUGE-2', 'Quora→WikiAnswers BLEU', 'Quora→WikiAnswers iBLEU', 'Quora→WikiAnswers ROUGE-1', 'Quora→WikiAnswers ROUGE-2']
In addition, we include iBLEU We use the same hyper-parameter in their original work. We compare DNPG with four existing neural-based models: ResidualLSTM Prakash et al. For a fair comparison, we also include a Transformer model with copy mechanism. In particular, the DNPG has similar performance with the vanilla Transformer model on Quora dataset, while significantly performs better on WikiAnswers. The reason maybe that the DNPG is more robust to the noise, since it can process the paraphrase in an abstractive way. It also validates our assumption that paraphrasing can be decomposed in terms of granularity. When the training data of high quality is available, the transformer-based models significantly outperforms the LSTM-based models. We evaluate the performance of the original DNPG and the Adapted DNPG in two settings of domain transfer: 1) Quora dataset as the source domain and WikiAnswers as the target domain, denoted as Quora→WikiAnswers, and 2) in reverse as WikiAnswers→Quora. For the baseline models, in addition to the pointer-generator network and the Transformer model with copy mechanism (denoted as Transformer+Copy), we use the shallow fusion Gulcehre et al. (MTL) Domhan and Hieber that harness the non-parallel data in the target domain for adaptation. For fair comparisons, we use the Transformer+Copy as the base model for shallow fusion and implement a variant of MTL with copy mechanism (denoted as MTL+Copy). DNPG achieves better performance over the pointer-generator and Transformer-based model, and has the competitive performance with MTL+Copy, which accesses target domain for training. With a fine-tuned separator, Adapted DNPG outperforms other models significantly on Quora→WikiAnswers. When it comes to WikiAnswers→Quora, where domain adaptation is more challenging since the source domain is noisy, the margin is much larger. The main reason is that the original meaning can be preserved well when the paraphrasing is conducted at the sentential level only. It is shown that the sentential paraphrasing is an efficient way to reuse the general paraphrase patterns and meanwhile avoid mistakes on rephrasing domain-specific phrases. However, it is at the expense of the diversity of the generated paraphrases. We leave this problem for future work.