Uncertainty-aware Self-training for Text Classification with Few Labels
Table 1: Dataset summary (W: avg. words / doc).
['Dataset', 'Class', 'Train', 'Test', 'Unlabeled', '#W']
Datasets. We perform large-scale experiments with data from five domains for different tasks as summarized in Table SST-2 (DBLP:conf/emnlp/SocherPWCMNP13), IMDB (DBLP:conf/acl/MaasDPHNP11) and Elec (DBLP:conf/recsys/McAuleyL13) are used for sentiment classification for movie reviews and Amazon electronics product reviews respectively. The other two datasets Dbpedia (DBLP:conf/nips/ZhangZL15) and Ag News (DBLP:conf/nips/ZhangZL15) are used for topic classification of Wikipedia and news articles respectively. Evaluation setting. For self-training, we fine-tune the base model (teacher) on K labeled instances for each task to start with. We also show results of the final model on varying K∈{20,30,50,100,500,1000}. We repeat each experiment five times with different random seeds and data splits, use the validation split to select the best model, and report the mean accuracy on the blind Test data. We implement our framework in Tensorflow and use four Tesla V100 gpus for experimentation. We use Adam (DBLP: journals/corr/KingmaB14) as the optimizer with early stopping and use the best model found so far from the validation loss for all the models. Hyper-parameter configurations with detailed model settings presented in Appendix.