Enriching Word Vectors with Subword Information
Table 3: Spearman’s rank correlation coefficient between human judgement and model scores for different methods using morphology to learn word representations. We keep all the word pairs of the evaluation set and obtain representations for out-of-vocabulary words with our model by summing the vectors of character n-grams. Our model was trained on the same datasets as the methods we are comparing to (hence the two lines of results for our approach).
['[EMPTY]', 'De Gur350', 'De ZG222', 'En WS353', 'En RW', 'Es WS353', 'Fr RG65']
We also compare our approach to previous work on word vectors incorporating subword information on word similarity tasks. The methods used are: the recursive neural network of \newciteluong2013better, the morpheme cbow of \newciteqiu2014colearning and the morphological transformations of \newcitesoricut2015unsupervised. In order to make the results comparable, we trained our model on the same datasets as the methods we are comparing to: the English Wikipedia data released by \newciteshaoul2010westbury, and the news crawl data from the 2013 WMT shared task for German, Spanish and French. We also compare our approach to the log-bilinear language model introduced by \newcitebotha2014compositional, which was trained on the Europarl and news commentary corpora. Again, we trained our model on the same data to make the results comparable. Using our model, we obtain representations of out-of-vocabulary words by summing the representations of character n-grams. We observe that our simple approach performs well relative to techniques based on subword information obtained from morphological segmentors. We also observe that our approach outperforms the \newcitesoricut2015unsupervised method, which is based on prefix and suffix analysis. The large improvement for German is due to the fact that their approach does not model noun compounding, contrary to ours.