Translating Neuralese
Table 1: Evaluation results for reference games. (a) The colors task. (b) The birds task. Whether the model human is in a listener or speaker role, translation based on belief matching outperforms both random and machine translation baselines.
['[EMPTY]', '[EMPTY]', 'as speaker R', 'as speaker H', '[EMPTY]']
The end-to-end trained model achieves nearly perfect accuracy in both cases, while a model trained to communicate in natural language achieves somewhat lower performance. This suggests that the automated agent has discovered a more effective strategy than the one demonstrated by humans in the dataset, and that the effectiveness of this strategy is preserved by translation.