Fluency-Guided Cross-Lingual Image Captioning
Table 5. Automated evaluation of six approaches to cross-lingual image captioning. Rejection sampling and weighted loss are comparable to ‘Without fluency’ which learns from the full set of machine-translated sentences.
['[BOLD] Approach', '[BOLD] Flickr8k-cn B-4', '[BOLD] Flickr8k-cn ROUGE', '[BOLD] Flickr8k-cn CIDEr', '[BOLD] Flickr30k-cn B-4', '[BOLD] Flickr30k-cn ROUGE', '[BOLD] Flickr30k-cn CIDEr']
The reranking strategy improves over ‘Late translation’ showing the benefit of fluency modeling. Nevertheless, both ‘Late translation’ and ‘Late translation rerank’ perform worse than the ‘Without fluency’ run. Fluency-only is inferior to other proposed approaches as this model is trained on much less amounts of data, more concretely, 2,350 sentences in Flickr8k and 15,100 sentences in Flickr30k that are predicted to be fluent. Both rejection sampling and weighted loss are on par with the ‘Without fluency’ run, showing the effectiveness of the two strategies for preserving relevant information.