Reconstruction of Word Embeddings from Sub-Word Parameters
Table 3: POS tagging accuracy with different definitions of vw (see the main text). The final column shows the number of lookup parameters.
['model', 'accuracy', 'lookup']
We see that pre-trained word embeddings boost the performance of full from 97.20 to 97.32. When we use the strictly character-based model char without reconstruction, the performance drops to 96.93. But with reconstruction, the model recovers some of the lost accuracy. In particular, reconstructing with the Manhattan distance metric gives the largeset improvement and yields 97.17.