Investigating Pretrained Language Models for Graph-to-Text Generation
Table 7: Impact of cross-domain supervised pretraining for BART-Base on KG-to-text generation.
['[BOLD] Pretrained on', '[BOLD] Evaluated on', '[BOLD] BLEU']
We also investigate how cross-domain pretraining affects the performance after fine-tuning. While the target texts in KGAIA and AGENDA share the domain of scientific abstracts , texts in WebNLG are more general. The graphs in all three datasets are KGs, but WebNLG graphs do not share any relations with the other KGs. Nevertheless, supervised pretraining increases the performance in the cross-domain setting in most of the cases. Only pretraining on the smaller WebNLG does not help for the larger AGENDA.