Self-Supervised Dialogue Learning
Table 3: The automatic evaluation of generated utterances on distinct-1 and distinct-2 metrics. Please refer to Section 4.2 Automatic Evaluation for explanations.
['[BOLD] Model', '[BOLD] distinct-1', '[BOLD] distinct-2']
, we use the two commonly accepted metrics distinct-1 and distinct-2. The distinct-1 and distinct-2, proposed by \newciteli2016diversity, are two ways to measure the degree of diversity by calculating the number of distinct unigrams and bigrams in the generated response utterances. The results show that based on the distinct-1 and distinct-2 metrics, the generator trained in our approach can generate relatively more diverse responses. The results are attractive considering that we do not explicitly use a diversity-guided objective function during the training process. We think the reason is that the diverse utterances are easier to reserve the order information. In previous methods, the discriminator D only gives good or bad signals to response generator G, and the G has to figure out what is an acceptable response by itself. As for our SSN, it explicitly forces the G to generate responses that will have unique orders in dialogue, which leads to more diverse utterances.