The Unbearable Weight of Generating Artificial Errors for Grammatical Error Correction
Table 1: (Exp2) Evaluating the impact of MLCONV, Transformer and the rule-based AEG systems. NUCLE-CLC column represents the F0.5 score of GEC models trained on the base NUCLE-CLC data. 10K, 50K, 100K, 500K, 1M, and 2M represents the amount of artificial data added to the NUCLE-CLC during training.
['[BOLD] GEC Model', '[BOLD] AEG model', '[BOLD] NUCLE-CLC', '[BOLD] 10K', '[BOLD] 50K', '[BOLD] 100K', '[BOLD] 500K', '[BOLD] 1M', '[BOLD] 2M']
Nearly all correction models improve when using MLCONV or Transformer AEG data with the biggest performances yielded using the Transformer model. Interestingly, when using 1M or 2M samples, performance starts to decline. We believe that over 1M samples, the noisiness of the artificial data overwhelms the contributions of the real data (roughly over 1M samples). The performance of all models drops when trained with the errors generated by the rule-based model. It is interesting to observe that the performance drops significantly just by adding 10K artificial sentences to the base data.