charagram: Embedding Words and Sentences via Character n-grams
Table 5: Results of using different numbers and different combinations of character n-grams.
['Task', '#  [ITALIC] n-grams', '2', '2,3', '2,3,4', '2,3,4,5', '2,3,4,5,6']
We performed a series of experiments to investigate how the charagram and charagram-phrase models perform with different numbers and lengths of character n-grams. For a given k, we took the top k most frequent character n-grams for each value of n in use. We experimented with k values in {100,1000,50000}. If there were fewer than k unique character n-grams for a given n, we used all of them. For these experiments, we did very little tuning, setting the regularization strength to 0 and only tuning over the activation function. We repeated this experiment for all three of our tasks. For word similarity, we report performance on SL999 after training for 5 epochs on the lexical section of PPDB XXL. For sentence similarity, we report the average Pearson’s r over all 22 datasets after training for 5 epochs on the phrasal section of PPDB XL. For tagging, we report accuracy on the validation set after training for 50 epochs.