Unsupervised Opinion Summarization with Noising and Denoising
Table 6: ROUGE-1/2/L F1 scores of our model and versions thereof with less synthetic data (second block), using only one noising method (third block), and without some modules (fourth block).
['[EMPTY]', 'Rotten Tomatoes', 'Rotten Tomatoes', 'Rotten Tomatoes', 'Yelp', 'Yelp', 'Yelp']
The final model consistently performs better on all metrics when compared to versions with less synthetic data (second block), versions with only one type of noise (second block), and versions with a module removed (third block). When using human-labeled categories, we see a slight improvement in ROUGE-2 on Rotten Tomatoes, however the model performs substantially worse on other metrics. We believe there are several reasons for this. Firstly, human-labeled categories, at least the ones available, are not fine-grained enough to capture various aspects mentioned in the reviews and their sentiment (e.g., did the actors perform well? was the plot convoluted?). Secondly, the number of business types available on Yelp is very large (i.e., 898 types), which makes the discriminator loss Ldisc hard to optimize. This explains the relatively larger decrease in performance on Yelp.