G-DAug: Generative Data Augmentation for Commonsense Reasoning
Table 6: Validation accuracy of G-DAug with different labeling methods on WinoGrande-L and CommonsenseQA. Random relabeling assigns labels uniformly at random, and model relabeling assigns labels according to a task model. Random labels hurt accuracy, and model relabeling helps on WinoGrande but not on CommonsenseQA.
['Method', 'WinoGrande-L', 'CSQA']
Even fully unsupervised language model pretraining can boost the performance of a task model, if performed on task-relevant data Gururangan et al. This raises the question of whether G-DAug boosts performance by simply exposing the model to more task-relevant text, or if the labels that G-DAug generates are in fact informative. A related question is whether G-DAug’s optional self-supervised relabeling improves performance. relabeling with a task model. When the generator labels are flipped randomly, G-DAug is unable to outperform the baselines for either dataset (and in fact dramatically underperforms the baseline on WinoGrande-L). This implies that the correctness of the labels plays an essential role in G-DAug. Relabeling with a task model provides a 1.5% absolute gain in WinoGrande-L, but a 0.4 point drop in CommonsenseQA.