DART: Open-Domain Structured Data Record to Text Generation
Table 5: The WebNLG 2017 results on the test set. †: We report results from Zhao et al. (2020a) who use the evaluation scripts that are strictly the same as the official challenge resulting in different numbers from the original papers. ‡: Results of our replications; we include previously unreported seen vs. unseen results from Moryossef et al. (2019b) by running evaluation scripts on their original output.
['[EMPTY]', 'BLEU ↑ SEEN', 'BLEU ↑ UNSEEN', 'BLEU ↑ ALL', 'METEOR ↑ SEEN', 'METEOR ↑ UNSEEN', 'METEOR ↑ ALL', 'TER ↓ SEEN', 'TER ↓ UNSEEN', 'TER ↓ ALL']
S4SS3SSS0Px2 WebNLG Furthermore, we also investigate if DART can be used to improve the performance on other data-to-text generation tasks. To this end, we finetune the BART model on the WebNLG 2017 challenge, and gradually augment the training data with the train split of DART. Our finetuned BART model achieves 39.05 BLEU scores on all categories, while adding DART as additional training data can improve it to 45.89. The improvement mainly comes from the unseen categories increasing from 26.23 to 37.85, demonstrating that DART is particularly helpful for out-of-domain generalization due to its open-domain nature.