A Multi-task Learning Approach for Improving Product Title Compression with User Search Log Data
Table 3: Manual evaluation results, including average core product recognition accuracy (Avg. Accu), average readability score (Avg. Read) and average informativeness score (Avg. Info).
['Method', 'Avg. Accu', 'Avg. Read', 'Avg. Info']
The core product recognition property is assessed with a binary score (i.e, 1 for correct and 0 for incorrect), and the other two properties are assessed with a score from 1 (worst) to 5 (best). To make annotation results more consistent and reliable, we exclude instances with divergent ratings (i.e., their variance is larger than a threshold). We conducted paired t-tests on the manual evaluation results. With respect to readability, our Agree-MTL is significantly (p < 0.05) better than all other methods. With respect to informativeness, our Agree-MTL is significantly (p < 0.05) better than all other methods except Ptr-Net (it would be significantly better than Ptr-Net when p < 0.08)