Zero-Resource Neural Machine Translation with Multi-Agent Communication Game
Table 4: Comparison the captioner’s performance of our proposed PRE. and JOINT model.
['Model', 'IAPR-TC12 img-de', 'IAPR-TC12 img-en', 'Multi30K img-de', 'Multi30K img-en']
We also compare the performance of the captioner with (JOINT) and without (PRE.) the joint training. Cheng et al. \shortciteCheng2017JointTF demonstrate that in pivot-based NMT, where the pivot is a third language, the source-pivot and pivot-target translation models can be improved simultaneously through joint training. In our setting, image-to-source captioning does not necessarily improve with joint training. It gets slightly worse on three out of four translation tasks. Using a third language as a pivot, the source-to-pivot and pivot-to-target models are symmetrical as they are both NMT models. In contrast, in our setting the captioner and the translator are not symmetrical anymore since NMT is much easier to improve than image caption. We suspect that the translator’s performance dominates the multi-agent game.