Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data
Table 6: Denoising Auto-encoder’s Results on CoNLL-2014 Test Data Set.
['[BOLD] Finetune', '[BOLD] Pre.', '[BOLD] Rec.', '[ITALIC] F0.5']
The results show, if we finetune the model for 1 epoch with the labeled training data, the pre-trained model beats the un-pretrained one with a big gap (48.89 vs 17.19). Even without finetune, the pre-trained model can get a F0.5 score of 31.33. This proves that pre-training gives the models much better initial parameters than the randomly picked ones.