Biases for Emergent Communication inMulti-agent Reinforcement Learning
Table 1: Both biases lead to consistent discovery of useful communication. We define a good run to be one with final average reward greater than 0.2. Averages are over 50 runs for each setting.
['Biases', 'Proportion of good runs', 'CI', 'Final reward of good runs']
When neither agent has any inductive bias, we see both Pl and Ps stay low throughout training, and the final reward of 0.1 is exactly what can be achieved in this environment with no communication. Importantly, when we add our inductive bias for positive listening, we see Ps increase initially, followed by Pl. This is consistent with the hypothesis that the positive listening bias bias produces a stronger learning signal for the speaker; then once the speaker has begun to learn to communicate meaningfully, the listener also has a strong learning signal. When we add the bias for positive signalling the reverse is true – Pl increases before Ps. This again fits the hypothesis that speaker’s bias produces a stronger learning signal for the speaker.