Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
Table 1: Performance of updated word vectors on the word analogy task. We split the English word analogy datasets (Mikolov et al., 2013a) into Out of vocab and In vocab questions. Out of vocab questions are composed of words that are out of vocabulary for S0, hence the null accuracy.
['[EMPTY]', 'Out of vocab', 'In vocab']
S4SS1SSS0Px4 Results. First, we see that fine-tuning X on S1 leads to decent performance on the Out of vocab questions, but saps the accuracy on In vocab questions. As mentionned before, learning vectors initialized with X on S1 may lead to a loss of important statistics learnt on S0: the total accuracy on In vocab questions drops by 3.3%.