Product-Aware Answer Generation in E-Commerce Question-Answering
Table 4. Consistency and fluency comparison by human evaluation.
['[EMPTY]', 'Fluency mean', 'Fluency variance', 'Consistency mean', 'Consistency variance']
For human evaluation, we ask annotators to rate each generated answer according to two aspects: consistency and fluency. The rating score ranges from 1 to 3, and 3 is the best. Although the BLEU score of S2SAR is lower than the S2SA, the embedding score and human score for S2SAR are higher than S2SA. Regardless of few word overlapping between generated and ground answer, the human evaluation and results in terms of embedding metrics verify S2SAR outperforms S2SA. This observation demonstrates the effectiveness of incorporating review in answer generation.