Compositional Languages Emerge in a Neural Iterated Learning Model
Table 5: Values of 4 metrics when |V| and NL changes. Metric G0.85 means the first generation that the average ρ of the previous three generations exceed 0.85. The notation “-” means the agents never satisfy the requirement.
['[EMPTY]', '[ITALIC] NL', '| [ITALIC] V|= 8', '| [ITALIC] V|=12', '| [ITALIC] V|=16', '| [ITALIC] V|=24', '| [ITALIC] V|=40', '| [ITALIC] V|=72']
From the discussion in Appendix B, we know that when |V| and NL are large, making high-ρ language dominate in the posterior probability is very hard, as the compositional languages only occupy an extremely small portion. Our algorithm, however, is very robust to different values of |V| and NL. An extreme example is that, the converged ρ can still be roughly 0.8 with |V|=72. The performance of validation accuracy seems more robust when |V| and NL changes: the NIL can always obtain more than 80% accuracy compared to the none reset case (roughly 15%). One possible explanation is that the increasing of NL brings an exponential change to the message space. However, no matter how |V| and NL change, EL∼P(L|NIL)[ρ(L)] is always significantly higher that the compositionality of emergent languages given by baseline model, i.e. 0.3.