Revisiting Paraphrase Question Generator using Pairwise Discriminator
Figure 3: Analysis of variants of our proposed method on QQP Dataset as mentioned in section 4.1.2. Summary results from different models for 100k dataset.Here L and P refer to the Local and Pairwise discriminator loss and S represents the parameter sharing between the discriminator and encoder module. G represents adversarial loss. ED represents Encoder-Decoder network. As we can see that our proposed method EDLPS clearly outperforms the other ablations on all metrics and detailed analysis is present in section 4.1.2.
['Model', 'BLEU_1', 'BLEU_2', 'BLEU_3', 'BLEU_4', 'ROUGE_L', 'METEOR', 'CIDEr']
Ablation Analysis We experimented with different variations for our proposed method. We start with a baseline model, which we take as a simple encoder and decoder network with only the local loss (EDL) as proposed by Sutskever et al. Sutskever et al. Further, we have experimented with encoder-decoder and a discriminator network with only global loss (EDP) to distinguish the ground truth paraphrase with the predicted one. Another variation of our model is used both the global and local loss (EDLP). The discriminator is the same as our proposed method, only the weight sharing is absent in this case. Another variation of our model is EDLPG, which has a discriminator and is trained alternately between local and global loss, similar to GAN training. Finally, we make the discriminator share weights with the encoder and train this network with both the losses (EDLPS). Among the ablations, the proposed EDLPS method works way better than the other variants in terms of BLEU and METEOR metrics by achieving an improvement of 7% and 3% in the scores respectively over the baseline method for QQP-I (50k) dataset and an improvement of 6% and 2% in the scores respectively for QQP-II (100k) dataset.