Long Short-Term Memory Over Tree Structures
Table 1: Performances (accuracies) of different models on the test set of Stanford Sentiment Tree Bank, at the sentence level (roots) and the phrase level. † shows the performance are statistically significantly better (p<0.05) than the corresponding models.
['Models', 'roots', 'phrases']
We present the results on 5-category sentiment prediction at both the sentence level (i.e., the ROOTS column in the table) and for all phrases including roots (the PHRASES column) As described earlier, we refer to recursive neural networks to as RvNN to avoid confusion with recurrent neural networks. RNTN is different from RvNN in that when merging two nodes to obtain the hidden vector of their parent, tensor is used to obtain the second-degree polynomial interactions. The S-LSTM results reported here were obtained by setting the size of the hidden units to be 100, batch size to be 10, and learning rate to be 0.1. In our experiments, we only tuned these hyper-parameters, and we feel that more finer tuning, such as discriminating the classification weights between the leaves (word embedding) and other nodes, using different numbers of hidden units for the memory blocks (e.g., for the hidden layers of words), or different initializations of word embedding, may further improve the performances reported here.