From Words to Sentences: A Progressive Learning Approach for Zero-resource Machine Translation with Visual Pivots
Table 2: The BLEU4 performance of different methods for image-pivoted zero-resource machine translation.
['[EMPTY]', '[EMPTY]', 'IAPR-TC12 De-En', 'IAPR-TC12 En-De', 'Multi30k De-En', 'Multi30k En-De']
Since our approach utilizes word translation learned in the first progressive step to benefit sentence translation from two losses, we compare each of the loss and their combination with previous methods. The Lpivot achieves superior performance to previous image-pivoted methods on both datasets and translation directions. It demonstrates the effectiveness of the re-weighting approach to suppress noises in image-pivoted sentence pairs, while such noises are ignored in previous methods. The auto-encoding loss Lae brings more significant performance gains, which shows that re-using word embedding matrices from the image-pivoted word translation is effective for the NMT model to encode language-agnostic sentence representation. The two losses are also complementary with each other and the combination of them achieves the best translation performance on the two datasets.