Unsupervised Opinion Summarization as Copycat-Review Generation
Table 8: Ablations, ROUGE scores on Amazon.
['[EMPTY]', 'R1', 'R2', 'RL']
To investigate the importance of the model’s individual components, we performed ablations by removing the latent variables (zi and c, one at a time), and attention over the other reviews. The models were re-trained on the Amazon dataset. They indicate that all components play a role, yet the most significant drop in ROUGE was achieved when the variable z was removed, and only c remained. Summaries obtained from the latter system were wordier and looked more similar to reviews. Dropping the attention (w/o r_i) results in more generic summaries as the model cannot copy details from the input. Finally, the smallest quality drop in terms of ROUGE-L was observed when the variable c was removed. In the introduction, we hypothesized that using the mean of latent variables would result in more “grounded” summaries reflecting the content of the input reviews, whereas sampling would yield texts with many novel and potentially irrelevant details.