MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification
Table 2: Performance (test accuracy(%)) comparison with baselines. The results are averaged after three runs to show the significance Dror et al. (2018), each run takes around 5 hours. Models are trained with 10, 200, 2500 labeled data per class. VAMPIRE, Bert, and TMix do not use unlabeled data during training while UDA and MixText utilize unlabeled data. * means our models.
['[BOLD] Datset AG News', '[BOLD] Model VAMPIRE', '[BOLD] 10 -', '[BOLD] 200 83.9', '[BOLD] 2500 86.2', '[BOLD] Dataset DBpedia', '[BOLD] Model VAMPIRE', '[BOLD] 10 -', '[BOLD] 200 -', '[BOLD] 2500 -']
All transformer based models (BERT, TMix, UDA and MixText) showed better performance compared to VAMPIRE since larger models were adopted. TMix outperformed BERT, especially when labeled data was limited like 10 per class. For instance, model accuracy improved from 69.5% to 74.1% on AG News with 10 labeled data, demonstrating the effectiveness of TMix. When unlabeled data was introduced in UDA, it outperformed TMix such as from 58.6% to 63.2% on Yahoo! with 10 labeled data, because more data was used and consistency regularization loss was added. Our proposed MixText consistently demonstrated the best performances when compared to different baseline models across four datasets, as MixText not only incorporated unlabeled data and utilized implicit relations between both labeled data and unlabeled data via TMix, but also had better label guessing on unlabeled data through weighted average among augmented and original sentences.