Reasoning about Pragmatics with Neural Listeners and Speakers
Table 2: Success rates at RG on abstract scenes. “Literal” is a captioning baseline corresponding to the base speaker S0. “Contrastive” is a reimplementation of the approach of Mao et al. (2015). “Reasoning” is the model from this paper. All differences between our model and baselines are significant (p<0.05, Binomial).
['Model', 'Dev acc.\xa0(%) All', 'Dev acc.\xa0(%) Hard', 'Test acc.\xa0(%) All', 'Test acc.\xa0(%) Hard']
Four randomly-chosen samples from our model. For each, the target image is shown on the left, the distractor image is shown on the right, and description generated by the model is shown below. All descriptions are fluent, and generally succeed in uniquely identifying the target scene, even when they do not perfectly describe it (e.g. (c)). Our reasoning model outperforms both the literal baseline and previous work by a substantial margin, achieving an improvement of 17% on all pairs set and 15% on hard pairs.