How agents see things:On visual representations in an emergent language game
Table 1: Percentage average rewards on the same-image, different-image and noise test sets for agents trained in the same- and different-image games (chance level at 50%). For each game, values are averaged on 10 test runs consisting of 1,000 games of mini-batches of 32 image pairs, using the cross-validated best seed.
['Train', '[EMPTY]', 'Test Same im.', 'Test Diff. im.', 'Test Noise']
We confirm that the same-image game is the easiest, and we observe that agents trained in one game perform reasonably well on the other. More importantly, no matter which game they are trained on, the agents perform very well on noise input! This confirms our hypothesis that the Sender and Receiver are able to communicate about input data that contain no conceptual content at all, which in turn suggests that they haven’t extracted any concept-level information (e.g., features that would allow them to recognize instances of the dog or chair category) during training.