Conversation Modeling on Reddit using a Graph-Structured LSTM
Table 4: Example comments and karma level predictions: reference, no text, graph(f), graph.
['Ex 1', 'karma 0', 'karma 7', 'karma 7', 'karma 0', 'Comment Republicans are fundamentally dishonest. (politics, id:1x9pcx)']
The first two examples are overpredicted (controversial) cases, where ignoring text leads to a high karma prediction, but the reference is zero. In the first case, the forward model incorrectly predicts high karma because “Republican” tends to be associated with positive karma. The model leveraging reply text correctly predicts the low karma. In the second case, the forward model captures reduces the prediction, but again having the replies is more helpful. The next two cases are examples of underprediction due to small subtrees. Example 3 is incorrectly labeled as level 0 by the forward and no-text models, but because the responses mention “nice joke” and “accurate analogy,” the bidirectional model is able to identify it as level 7. Example 4 has only one child, but both models using language correctly predict level 7, probably because the model has learned that references to “Colbert” are popular. The next two examples are underpredicted cases from early in the discussion, many of which expressed an opinion that in some way provided multiple perspectives. Finally, the last two examples represent instances where neither model successfully identifies a high karma comment, which often involve analogies. Unlike the “titanic” analogy, these did not have sufficient cues in the replies.