Open Domain Dialogue Generation with Latent Images
Table 4: Results on topical metrics.
['Models', 'Seq2Seq', 'HRED', 'VHRED', 'ReCoSa', 'ImgVAE']
S4SS5SSS0Px2 Answer to Q2 : we define two new metrics in text-based conversation: topic=1|DtT|∑(c,r)∈DtT|τ(rg)|l(rg),novel−topic=1|DtT|∑(c,r)∈DtT|τ(rg)−τ(c)|l(rg), (14) where DtT refers to the test set of the Reddit data, ( c,r) is a context-response pair, rg is a response generated accroding to c, τ(x) returns a set of topical words in sentence x, |⋅| measures the size of a set, and l(x) returns the length of x. topic measures the average proportion of topical words in generated responses, while novel-topic further excludes topical words appearing in contexts. We can see that the latent images significantly enhance the ratio of topical words and the ratio of extra topical words in responses. Seq2Seq has a high topic score but the lowest novel-topic score, because it tends to copy words from contexts in response synthesis. topic and novel-topic for human responses in the test set are 0.398 and 0.321 respectively. This means that even though ImgVAE can enhance appearance of informative content, it is still not so good as humans at bringing in new content and thus extending conversation, which could be a future direction for informative response generation.