Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems
Table 2: Evaluation on MultiWOZ with the nucleus sampling procedure.
['[EMPTY]', 'Baseline', 'GPT', 'GPT2-S', 'GPT2-M']
The scores confirm the importance of choosing the correct sampling method. The GPT2 models improve the score on Inform and Success metrics. It is worth noting the consistent drop in BLUE scores across all models. This comes from the fact that nucleus sampling allows for increased variability: this might reduce the probability of generating domain-specific tokens.