Learning to generate one-sentence biographies from Wikidata
Table 6: Fact-set content selection results phrased as precision, recall and F1 of systems with respect to the Wikipedia reference on dev.
['[EMPTY]', 'P', 'R', 'F']
Do systems select the same facts found in the reference summaries? This suggests that the autoencoding in s2s+ae helps increase fact recall without sacrificing precision. The template baseline also attains this higher recall, but at the cost of precision. For commonly expressed facts found in most person biographies, recall is over 0.95 (e.g., CITIZENSHIP, BIRTH_DATE, DEATH_DATE and OCCUPATION). Facts that are infrequently expressed are more difficult to select, with system F1 ranging from 0.00 to 0.50. Interestingly, macro-averaged F1 across infrequently expressed facts mirror human preference rather than bleu results, with s2s+ae (0.26) > base (0.17) > s2s (0.07). However, all systems perform poorly on these facts and no reliable differences are observed.