Attractive or Faithful? Popularity-Reinforced Learning for Inspired Headline Generation
Table 4: Attractiveness evaluation results.
['Models', 'Attractiveness']
We statistically analyze the attractiveness and faithfulness of the headlines. For the extraction, the performances of IRBM25 and random models are poor, which suggests that memorizing the whole training corpus or randomly picking does not work. In contrast, the PREFIX model performs quite well, which is expected given that most news articles state the key points in the first sentence. The extraction result of PORL-HG shows that the way we incorporate the popularity information does not affect the ROUGE score. The Meteor metric is used for faithfulness comparison, which is calculated by the recall and the precision, and takes synonyms into consideration. The PORL-HG significantly outperforms the baseline by 27.60% for attractiveness while slightly improves the baseline in terms of Meteor. Moreover, the PTA contributes the most for attractiveness (from 40.76 to 44.06), suggesting that simultaneously considering topic distributions of sentences and popular headlines is effective.