Data Distillation for Controlling Specificity in Dialogue Generation
Table 5: Pairwise human judgments between the reinforcement learning model (RL) and the first two distillation models (Iter1 and Iter2).
['pairs', 'win', 'lose', 'tie']
We present labelers with an input message and the generated outputs from three models, Iter1, Iter2, and RL, and ask them to rank the three outputs by quality. Note that the outputs from the RL model can be the same as those from Iter1 or Iter2 if the RL model chooses that particular model (Iter1 or Iter2) for decoding. In these cases, a tie is automatically recorded. As can be seen, the reinforcement learning model performs best 60 percent of the time, followed by Iter2, which wins 37 percent of the time. It is interesting to see that Iter2 generally outperforms Iter1, winning on 62 percent of the examples. This is consistent with the fact that the RL model tends to prefer Iter2 more often.