Retrieve and Refine: Improved Sequence Generation Models For Dialogue
Table 1: Perplexity on the ConvAI2 task test set with different types of retriever for RetNRef, see text.
['RetNRef Retrieval Method', 'PPL']
They show that the RetNRef model can indeed improve perplexity with label neighbors or the label itself. However, surprisingly there is almost no difference between using no retrieval, random labels or our best retriever. The RetNRef++ model – that truncates the dialogue history and focuses more on the retrieval utterance – does even worse in terms of perplexity: 48.4 using the Memory Network retriever. However, poor perplexity does not mean human judgments of the generated sequences will not improve; in fact we will see that they do in the next section. How to automatically evaluate these kinds of models still remains an open problem.