Triple-to-Text: Converting RDF Triples into High-Quality Natural Languages via Optimizing an Inverse KL Divergence
Table 6. Human evaluation on WebNLG dataset.
['methods', 'Grammar', 'Correctness']
Human evaluation is conducted on WebNLG dataset to validate the performance of our framework further. We choose WebNLG dataset because it consists of more RDF triples and its reference sentences are relatively simple. We randomly select 20 RDF triples from the dataset, along with the corresponding sentences generated by T2T and baselines. Ten human volunteers are asked to rate the sentences from two aspects: grammar and correctness. The score on grammar is used to judge whether the sentence contains grammatical errors, improper use of words and repetition. Correctness measures whether the sentence accurately represents the information in the RDF triples. The score for each criterion takes an integer between 1 and 10. Volunteers are given both scoring criteria and examples.