Recursive Graphical Neural Networks for Text Classification
Table 5: Ablation study on R52 and Reuters21578. It is clear that LSTM plays an important role by alleviating over-smoothing problem, especially in multi-label classification, which is more prone to over-smoothing.
['Model', 'R52', 'Reuters21578']
From the results we can see that removing any of the three parts of our proposed model would lead to a decline in accuracy. Among the three parts, the accuracy of the model without LSTM decreases most significantly. We assume that this is because that the over-smoothing problem becomes very severe with relatively big layer number. Furthermore, compared with multiple label classification, smoothing is more acceptable for single label classification. Because if the representation of all the nodes grow to be related to the correct label, it will not hurt the performance. However, over-smoothing is more harmful for multi-label problem, because multi-label requires the representation of different parts related to different labels to be distinguishable.