Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach
Table 1: Evaluation results. Marked with asterisks are individual results higher than the ZSDG baseline which are achieved with the minimum amount of training data, and in bold is the model consistently outperforming ZSDG in all domains and metrics with minimum data.
['[BOLD] ModelDomain', '[BOLD] Navigation BLEU, %', '[BOLD] Navigation Entity F1, %', '[BOLD] Weather BLEU, %', '[BOLD] Weather Entity F1, %', '[BOLD] Schedule BLEU, %', '[BOLD] Schedule Entity F1, %']
Our objective here is maximum accuracy with minimum training data required, and it can be seen that few-shot models with LAED representation are the best performing models for this objective. While the improvements can already be seen with simple FSDG, the use of LAED representation helps to significantly reduce the amount of in-domain training data needed: in most cases, the state-of-the-art results are attained with as little as 3% of in-domain data. At 5%, we see that FSDG+LAED consistently improves upon all other models in every domain, either by increasing the mean accuracy or by decreasing the variation. In contrast, the ZSDG setup used approximately 150 annotated training utterances for each of the 3 domains, totalling about 450 annotated utterances. Although in our few-shot approach we use full in-domain dialogues, we end up having a comparable amount of target-domain training data, with the crucial difference that none of those has to be annotated for our approach. Therefore, the method we introduced attains state-of-the-art in both accuracy and data-efficiency.