Emergence of Numeric Concepts in Multi-Agent Autonomous Communication
Table 4.6: ρ and p-values of different types of languages. ρ represents the Spearman correlation coefficient between meanings and messages, and the “Emergent-R” and “Emergent-S” are emergent languages in Set-Reconstruct game and Set-Select game respectively.
['Language Type', 'BLEU-1  [ITALIC] ρ', 'BLEU-1  [ITALIC] p-value', 'BLEU-2  [ITALIC] ρ', 'BLEU-2  [ITALIC] p-value', 'BLEU-3  [ITALIC] ρ', 'BLEU-3  [ITALIC] p-value']
To test whether there is a correlation between BLEU scores and sharing same numeric concepts, we calculate the Spearman correlation coefficient and the corresponding p-values. The results got on different types of languages based on The p-values under BLEU-1/2/3 for compositional language as well as emergent languages in both Set-Reconstruct and Set-Select games are all smaller than 0.05. Thus, it is safe to reject null hypothesis and accept the alternative hypothesis. That is, The BLEU scores between messages are highly correlated with whether their meaning pairs share same numeric concepts.