Examining Cooperation in Visual Dialog Models
Table 2: MPR [%] of Negation intervention results.
['[BOLD] Round', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
We start intervening from round 0, 2, 4, 6, and 8. Comparing the MPR across all rounds (columns in the table), we see that Q-bot is sensitive to the non-cooperative behaviors of A-bot, however only to a small degree (< 0.3%). Our results suggest that in addition to downstream evaluation in a cooperative game setting, forcing one agent to play non-cooperatively could help researchers design a better experimental setup to understand the cooperative behaviours amongst agents in the system.