The CUED’s Grammatical Error Correction Systems for BEA-2019
Table 8: Using back-translation for GEC (Base models). The third column contains the ratio between real and synthetic sentence pairs.
['[BOLD] Over-sampling Rate  [BOLD] (Real Data)', '[BOLD] Number of  [BOLD] Synthetic Sentences', '[BOLD] Ratio', '[BOLD] CoNLL-2014  [BOLD] P', '[BOLD] CoNLL-2014  [BOLD] R', '[BOLD] CoNLL-2014  [BOLD] M2', '[BOLD] BEA-2019 Dev  [BOLD] P', '[BOLD] BEA-2019 Dev  [BOLD] R', '[BOLD] BEA-2019 Dev  [BOLD] ERRANT']
Our experiments with back-translation are summarized in Tab. Adding 1M synthetic sentences to the training data already yields very substantial gains on both test sets. We achieve our best results with 5M synthetic sentences (+8.44 on BEA-2019 Dev). Over-sampling the real data is a common practice to rectify that ratio if large amounts of synthetic data are available. Interestingly, over-sampling real data in GEC hurts performance (row 3 vs. 5 in Tab. (last three rows in Tab. We will proceed with the 5M setup for the remainder of this paper.