Neural Syntactic Preordering for Controlled Paraphrase Generation
Table 1: Quality and diversity metrics for the different models. Our proposed approach outperforms other diverse models (SCPN and diverse decoding) in terms of all the quality metrics. These models exhibit higher diversity, but with many more rejected paraphrases, indicating that these models more freely generate bad paraphrases.
['Model', 'oracle quality (over 10 sentences, no rejection) ↑ BLEU', 'oracle quality (over 10 sentences, no rejection) ↑ ROUGE-1', 'oracle quality (over 10 sentences, no rejection) ↑ ROUGE-2', 'oracle quality (over 10 sentences, no rejection) ↑ ROUGE-L', '% rejected', 'pairwise diversity (post-rejection) self-BLEU ↓', 'pairwise diversity (post-rejection) self-WER ↑']
The results show that our proposed model substantially outperforms the SCPN model across all quality metrics. Furthermore, our LSTM model also beats the performance of the SCPN model, demonstrating that the gain in quality cannot completely be attributed to the use of transformers. The quality of our full model (with rearrangements) is also comparable to the quality of the vanilla seq2seq model (without rearrangements). This demonstrates that the inclusion of rearrangements from the syntax-based neural transducer do not hurt quality, while leading to a substantially improved diversity performance.