Better Early than Late: Fusing Topics with Word Embeddingsfor Neural Question Paraphrase Identification
Table 3: Ablation study for our TAPA model reporting F1 scores on test sets.
['[EMPTY]', 'PAWS', 'Quora', 'Sem-Eval']
Removing topics consistently reduces F1 scores on all datasets, while the effect of ELMo representations is dataset dependent. Deleting ELMo improves performance on Quora, but leads to a massive performance drop on PAWS. The large impact on PAWS can be explained by the fact that this dataset was automatically constructed to have high textual overlap between questions and differences between paraphrases are chiefly due to variations in syntax. Our full TAPA model uses early fusion as this was the best setting during hyperparameter tuning. We conclude that topics contribute consistently to the performance of our proposed model, but that early topic-embedding fusion is crucial.