Text-to-Text Pre-Training for Data-to-Text Tasks
Table 5: Results on Multiwoz.
['Model', 'BLEU', 'SER']
All T5 based models (including T5-small which has 5x fewer parameters) outperform SC-GPT by 4-5 BLEU without any in-domain pre-training. While the SER scores are slightly worse, upon manual inspection we found that the difference can largely be attributed to false positives arising from a combination of annotation inconsistencies in the dataset coupled with the exact match constraint, which does not account for paraphrases.