Few-Shot Natural Language Generation by Rewriting Templates
Table 2: Performance of models on MultiWOZ.
['[EMPTY]', '[BOLD] BLEU', '[BOLD] SER']
Naive achieves a BLEU score of 34.96 and outperforms the previous best model SC-GPT by 4+ points, setting a new state of the art. SC-GPT consists of a GPT-2 model further pre-trained on a large in-domain NLG corpus. On the other hand, we do not use any such corpus and directly fine-tune on T5. While the SER score is slightly higher, we found that most of the errors can be attributed to the noisy string matching aspect of the metric and were not actually errors. T2G2 performs on par with Naive. This is likely due to the large size of the MultiWOZ dataset (57K utterances spread over just 5 domains) and indicates that with enough annotated data, a simple pre-train and fine-tune approach is enough to attain good performance. Few shot and zero shot settings offer a greater and more realistic challenge, and we explore these settings next.