Meshed-Memory Transformer for Image Captioning
Table 5: Performances on nocaps validation set, for in-domain and out-of-domain captioning.
['[EMPTY]', 'In-Domain CIDEr', 'In-Domain SPICE', 'Out-of-Domain CIDEr', 'Out-of-Domain SPICE', 'Overall CIDEr', 'Overall SPICE']
We also assess the performance of our approach when dealing with images containing object categories that are not seen in the training set. We compare with Up-Down [anderson2018bottom] and Neural Baby Talk [lu2018neural], when using GloVe word embeddings and Constrained Beam Search (CBS) [anderson2016guided] to address the generation of out-of-vocabulary words and constrain the presence of categories detected by an object detector. To compare with our model, we use a simplified implementation of the procedure described in [agrawal2019nocaps] to extract constraints, without using word phrases. As expected, the use of CBS significantly enhances the performances, in particular on out-of-domain captioning.