Investigating Pretrained Language Models for Graph-to-Text Generation
Table 3: Results on WebNLG test set. A, S and U stands for all, seen and unseen partitions of the WebNLG test sets, respectively.
['[BOLD] Model', '[BOLD] BLEU  [BOLD] A', '[BOLD] BLEU  [BOLD] S', '[BOLD] BLEU  [BOLD] U', '[BOLD] METEOR  [BOLD] A', '[BOLD] METEOR  [BOLD] S', '[BOLD] METEOR  [BOLD] U', '[BOLD] chrF++  [BOLD] A', '[BOLD] chrF++  [BOLD] S', '[BOLD] chrF++  [BOLD] U']
We compare the performance of our experiments with many state-of-the-art results reported on this dataset. Melbourne is one of the best competitors in the WebNLG challenge. moryossef-etal-2019-step and castro-ferreira- etal-2019-neural develop neural pipeline approaches with explicit intermediate steps in the generation process and achieve strong performance in the unseen dataset. On the other hand, fully end-to-end models ribeiro2020modeling; schmitt2020modeling have strong performance on seen dataset and usually perform poorly in unseen data. A very recent model zhao-etal-2020-bridging achieves the best performance in the seen test set. However, this model leverages additional information about the order that the triples are realized into the target text. T5 achieves 59.29 and 54.69 BLEU points on all seen and unseen datasets. Note that kale2020texttotext also employ T5 in this benchmark. BART performs much worse than T5 on this dataset, with around 10 BLEU points lower compared to T5 in all seen partition.