Focus on What’s Informative and Ignore What’s not: Communication Strategies in a Referential Game
Table 1: Top: results of the experiments performed with the uniform data distribution (U) and with the skewed one (S). Unique target vectors and unique msgs represent the distinct input vector count seen by the sender and the unique symbols count produced, respectively. H(msgs) is the entropy of the emergent protocol, the higher the value the more the message distribution approaches a uniform one. H(vectors) is the entropy of the target vectors. Bottom: mutual information and standard deviation between each target vector feature and and the message distribution.
['Data', 'Acc.', 'Unique Msgs', 'H(msgs)', 'Unique Target Vectors', 'H(targets)']
While showing good performance, the agents do not reach perfect accuracy, suggesting that they are not making full use of the feature space. Indeed, although MI has similar average values across features, we constantly observed that, in each run, three of them had a higher MI compared to the remaining two, in line with our hypothesis that the agents rely on an arbitrary subset of features to describe the target vector. In this way, they pay a small accuracy cost in exchange for lower protocol complexity. In the uniform setup, the choice of the feature subset is arbitrary. We looked next at whether we can influence it by making a feature less informative than the others. We see that accuracy is comparable in both uniform and non-uniform conditions. The unique target vectors are less in the skewed configuration than in the uniform one. We also found a lower entropy in this configuration as having a non-uniform feature makes sampling the same object more likely. The number of unique messages produced, as well as their entropy, is only very slightly lower in the skewed configuration (8% less message for 30% less target vectors), making us think that messages are used effectively. Similarly to the uniform environment, also in the the skewed configuration we see close average values of MI across all the uniform features, although, again, in specific runs 3 arbitrary features would have particularly high MIs. We confirm that making one feature in the environment non-uniform leads the agents to ignore it (MI value of 0.07). More surprisingly, we see higher average MI values for the remaining features compared to the uniform simulation (features 2, 3 and 4 show a 30% MI increase, feature 5 a 26% increase). This suggests that making one feature less informative has led the agents to make a better use of the other features, compared to when they have to choose from a larger set of equally informative ones.