Data-to-text Generation with Entity Modeling
Table 1: Vocabulary size, number of tokens, number of instances (i.e., record-summary pairs), average summary length, number of record types and average number of records in RotoWire and MLB datasets.
['[EMPTY]', 'RotoWire', 'MLB']
We performed experiments on two datasets. The first one is RotoWire Wiseman et al. In addition, we created MLB, a new dataset which contains baseball statistics and corresponding human-authored summaries obtained from the ESPN website. As can be seen, MLB is approximately five times larger than RotoWire, with richer vocabulary and longer summaries. For RotoWire, we used the official training, development, and test splits of 3,398/727/728 instances. Analogously, for MLB we created a split of 22,821/1,739/1,744 instances. Game summaries in MLB were tokenized using nltk and hyphenated words were separated. Sentences containing quotes were removed as they included opinions and non-factual statements unrelated to the input tables. Sometimes MLB summaries contain a “Game notes” section with incidental information which was also removed. As far as MLB is concerned, ENT achieves highest scores on Grammaticality and Coherence. It is rated high on Conciseness also, second only to TEMPL whose scores are lowest on Grammaticality and Coherence. Perhaps surprisingly, Gold is rated lower than ENT on all three metrics; we hypothesize that participants find Gold’s output too verbose compared to the other systems. The average length of output summaries for ENT is 327 tokens.