Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
Table 2: Classification accuracy on Yelp reviews from 2018. We compare models trained on 1.2M reviews from 2013-2014 (S0) and a those trained on a smaller sample of reviews from 2018 (S1). We vary the size of S1 from 10k to 500k samples, while keeping S0 of fixed size.
['[EMPTY]', 'Size of  [ITALIC] S1 10k', 'Size of  [ITALIC] S1 30k', 'Size of  [ITALIC] S1 100k', 'Size of  [ITALIC] S1 500k']
First of all, we observe that the performance of models trained on S0 and S1 strongly depend on the size of S1. When the two datasets are of the same size (500k), the best performing model is the one trained on S1, as in that case there is no train/test distribution discrepancy. However, when S1 is small (10k or 30k), it is better to use a larger yet ill-distributed dataset (70.8% versus 74.9%). We also observe that when training on a concatenation of the two, the model performs at least as well as the best one.