Char-RNN and Active Learning for Hashtag Segmentation
Table 5: Accuracy of LSTM char-RNN on both Russian and English datasets
['[EMPTY]', 'Accuracy, Russian dataset', 'Accuracy, English dataset']
In our experiments we used 5 epochs to train the char-RNN with LSTM units. For each language we observed three datasets with different number of hashtags. In case of Russian language, the more data we use while training, the higher the accuracy. Due to it’s lower morphological diversity and complexity the model starts to overfit on training sets with large sizes. The training showed that mostly the model makes wrong predictions of segmentation on hashtags of complex types, such as ‘‘wordword_worddigits’’.