Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation
Table 1: Performance on WikiBio test set. Two Fluency measures differ in whether to include sentences graded as Mostly Fluent. Starred numbers are statistically significant against baselines (p
['[BOLD] Model', 'Automatic Metrics  [BOLD] BLEU', 'Automatic Metrics  [BOLD] PARENT (Prec. / Rec. / F1)', 'Automatic Metrics  [BOLD] Avg Len.', 'Human evaluation  [BOLD] Faithful %', 'Human evaluation  [BOLD] Avg Cov.', 'Human evaluation  [BOLD] Fluency %']
According to human evaluation, our approach gives a clear improvement in faithfulness over the baselines, with some drop in coverage. To further measure the validity of our confidence score, we postprocessed the output to remove words with lower confidence than 0.125. This thresholding technique gives further gains to faithfulness, while sacrificing some fluency.