Learning Data Manipulation for Augmentation and Weighting
Table 1: Accuracy of Data Manipulation on Text Classification. All results are averaged over 15 runs ± one standard deviation. The numbers in parentheses next to the dataset names indicate the size of the datasets. For example, (40+2) denotes 40 training instances and 2 validation instances per class.
['[EMPTY]', '[BOLD] Model', '[BOLD] SST-5 (40+2)', '[BOLD] IMDB (40+5)', '[BOLD] TREC (40+5)']
For data augmentation, our approach significantly improves over the base model on all the three datasets. Besides, compared to both the conventional synonym substitution and the approach that keeps the augmentation network fixed, our adaptive method that fine-tunes the augmentation network jointly with model training achieves superior results. Indeed, the heuristic-based synonym approach can sometimes harm the model performance (e.g., SST-5 and IMDB), as also observed in previous work (wu2018conditional; kobayashi2018contextual). This can be because the heuristic rules do not fit the task or datasets well. In contrast, learning-based augmentation has the advantage of adaptively generating useful samples to improve model training. Our weight learning consistently improves over the base model and the latest weighting method (ren2018learning). In particular, instead of re-estimating sample weights from scratch in each iteration (ren2018learning), our approach treats the weights as manipulation parameters maintained throughout the training. We speculate that the parametric treatment can adapt weights more smoothly and provide historical information, which is beneficial in the small-data context. Our approach provides the generality to instantiate diverse manipulation types and learn with the same single procedure.