Few-Shot Natural Language Generation by Rewriting Templates
Table 5: Results of human evaluation. † Significantly better than human. ∗ Significanly better than SlotDesc. Statistical significance computed using a one tailed t-test, p < 0.01
['[BOLD] Naturalness', '[BOLD] Naturalness Naive', '[BOLD] Naturalness SlotDesc', '[BOLD] Naturalness T2G2', '[BOLD] Naturalness Human']
Somewhat surprisingly, for seen domains even the baseline model performs on par with the ground truth. This is in line with recent work on task oriented NLG Peng et al. For unseen domains, T2G2 provides large improvements over baselines for both informativeness and naturalness, confirming the trends from automatic improvements. Remarkably, T2G2 also outperforms the human authored ground truth responses. We take this as a promising indication of the real world applicability of our approach.