Query-Variant Advertisement Text Generation with Association Knowledge
Table 3: Human evaluation results. “win” indicates the generated advertisement of our proposed model is better than the other one. “win>lose” means our proposal is better than the other one. The Pearson’s r for each metric is shown in the header within the braces. “NoQuery” means not using queries when doing inference.
['Versus', 'Attractiveness (p=0.44) win', 'Attractiveness (p=0.44) lose', 'Attractiveness (p=0.44) tie', 'Informativeness (p=0.46) win', 'Informativeness (p=0.46) lose', 'Informativeness (p=0.46) tie', 'Fluency (p=0.22) win', 'Fluency (p=0.22) lose', 'Fluency (p=0.22) tie']
Result: We compare our model with the three baselines, namely, the original human written advertisement, Transformer and NoQuery. NoQuery means the result is generated by the model we propose without adding query in the test phase. When “win > lose”, our model beats the other one. Results show that our proposed model beats all the baselines, especially in attractiveness and informativeness. The evaluation results on attractiveness and informativeness show good correlation, while rather low correlation on fluency. This is expected because the concept of being fluent is blurry especially in advertisement area.