Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input
Table 2: Communicative success (acc in percentage) of agents evaluated on training (first row) and novel (last three rows) data. lexicon size column reports the percentage of novel messages (i.e., messages that were not used during the training).
['[BOLD] Data', '[BOLD] length 2  [BOLD] lexicon size', '[BOLD] length 2  [BOLD] acc.', '[BOLD] length 5  [BOLD] lexicon size', '[BOLD] length 5  [BOLD] acc.', '[BOLD] length 10  [BOLD] lexicon size', '[BOLD] length 10  [BOLD] acc.']
While there is a drop in performance for unseen objects, agents are performing above random chance (20%). The emerged language is indeed able to generalize to unseen objects; however, the degree of generalization is a function of the similarity between the training and unseen objects, thus resulting in the uniform chimeras setting obtaining the lowest performance. Moreover, we observe examples of productivity, a key feature of compositionality. At test time, speakers are able to concoct novel messages on-the-fly (i.e., messages that are not part of their lexicon induced during training) to describe unseen objects. Even though listeners were not trained to associate novel messages with novel objects, they are still able to comprehend such messages and correctly identify the target object. In the test data and length 10 cases, novel messages account for almost all of the generated messages, but with performance at 81.6%, providing evidence of the structure found in the messages.