Stochastic Natural Language Generation Using Dependency Information
Table 5: Results of Human Evaluations on three tabular format datasets in terms of Informativeness, Naturalness and Quality, rating out of 5. For baseline we used gold-standard references in each dataset.
['[BOLD] Dataset', '[BOLD] Informativeness  [BOLD] Ours', '[BOLD] Informativeness  [BOLD] Baseline', '[BOLD] Naturalness  [BOLD] Ours', '[BOLD] Naturalness  [BOLD] Baseline', '[BOLD] Quality  [BOLD] Ours', '[BOLD] Quality  [BOLD] Baseline']
Since in our proposed system all meaning labels are forced to exist in the produced dependency tree, the mean score for informativeness factor is 5, except for RoboCup dataset; because in this dataset, each sentence corresponds to one meaning label and 60% of the training sentences are corresponding to PASS record type. So, as a result, at the test time, since test scenarios consist of multiple record types including PASS, a few of the sentences, that have less training sample compare to the other record type (like Offside or BallStopped that have less than 2% training sample in total), are wrongly generated for this record.