Generating Diverse Translation by Manipulating Multi-Head Attention
Table 5: Pair-wise BLEU and Reference BLEU in En2Ro experiments.
['Model', 'rfb\\uparrow', 'pwb\\downarrow', 'DEQ\\uparrow']
\citeauthorLi2016ASF \shortciteLi2016ASF bring very limited enhancement in diversity, failing to achieve the goal. \citeauthorVijayakumar2018DiverseBS \shortciteVijayakumar2018DiverseBS and \citeauthorShen2019MixtureMF \shortciteShen2019MixtureMF show the ability to produce diversity, but our method (K=4) attains more significant diversity as well as better quality comparing with them. And K=3 achieve the highest DEQ, gaining the most satisfactory result. Also, unlike \citeauthorShen2019MixtureMF \shortciteShen2019MixtureMF, our work needs no extra training or extra parameters. Furthermore, the diversity can be well interpreted and does not rely on an abstract latent variable.