Towards Lossless Encoding of Sentences
Table 2: SST-5 and SST-2 performance on all and root nodes respectively. Model results in the first section are from the Stanford Treebank paper Socher et al. (2013). GenSen and BERTBASE results are from Subramanian et al. (2018) and Devlin et al. (2018) respectively.
['Model', 'SST-5 (All)', 'SST-2 (Root)']
For the five class sentiment task, we compare our model with the original Sentiment Treebank results and beat all the models. In order to compare our approach with state-of-the-art methods, we also trained our model on the binary classification task with sole classification of the root node. Other presented models are GenSen Subramanian et al. Both these recent methods perform extremely well on multiple natural language processing tasks. We set the RAE embedding size demb to 1024. Larger embedding sizes did not improve the accuracy of our model for this task. In this setting, the RAE has 11M parameters, while the models we compare with, GenSen and BERTBASE, have respectively 100M and 110M parameters. Both our model and GenSen fail to beat the RNTN model for the SST-2 task. We see an improvement in accuracy when combining both methods’ embeddings, surpassing every model in the SST paper, while being close to BERTBASE’s performance.