Zero-Shot Dialog Generation with Cross-Domain Latent Actions
Table 1: Evaluation results on test dialogs from SimDial Data. Bold values indicate the best performance.
['[BOLD] In domain', '+Attn', '+Copy', '+Attn +AM', '+Copy +AM']
Although the standard +Attn model achieves good performance in the source domains, it doesn’t generalize to target domains, especially for entity F1 in the unseen-slot domain, BLEU score in the unseen-NLG domain, and all new domain metrics. The +Copy model has better, although still limited, generalization to target domains. The main benefit of the +Copy model is its ability to directly copy and output words from the context, reflected in its strong entity F1 in the unseen slot domain. However, +Copy can’t generalize to new domains where utterances are novel, e.g. the unseen NLG or the new domain. However, our AM algorithm substantially improves performance of both decoders (Attn and Copy). Results show that the proposed AM algorithm is complementary to decoders with a copy mechanism: HRED+Copy+AM model has the best performance on all target domains. In the easier unseen-slot and unseen-NLG domains, the resulting ZSDG system achieves a BEAK of about 82, close to the in-domain BEAK performance (84.7). Even in the new domain (movie), our model achieves a BEAK of 67.2, 106% relative improvement w.r.t +Attn and 38.8% relative improvement w.r.t +Copy. Moreover, our AM method also improves performance on in-domain dialogs, suggesting that AM exploits the knowledge encoded in the domain description and improves the models’ generalization.