Comment Ranking Diversification in Forum Discussions
Table 2: Comparison of various comment embedding methods. Median quantile difference computes the difference in average cosine similarity rank (percentile) of Gold 1 pairs - Gold 0 pairs. Logistic regression predicts the accuracy of the gold labels trained using each model’s pairwise cosine similarity matrix as input.
['[BOLD] Embedding Method', '[BOLD] Median Quantile Difference', '[BOLD] Logistic Regression Accuracy']
Using the best performing model for these two metrics, comment similarity was computed using cosine similarity [huang2008similarity]. For our task, word-level comment embedding methods (word2vec, Gated CNN, LDA) performed worse than a simple TFIDF vector representation alone, with a classical application of dimensionality reduction using PCA achieving highest accuracy on our gold dataset. In the rest of this section, we discuss potential reasons for this.