Non-autoregressive Transformer by Position Learning
Table 2: Results on validation set and test set of Quora.
['Model', 'Paraphrase(BLEU) Valid', 'Paraphrase(BLEU) Test']
In consist with our intuition, PNAT achieves the best result on this task and even surpass Transformer around 3.5 BLEU. The NAT model is not powerful enough to capture the latent position relationship. The comparison between NAT-base and PNAT shows that explicit position modeling in PNAT plays a crucial role in generating sentences.