Parallel Data Augmentation for Formality Style Transfer
Table 2: The comparison of different data augmentation methods for FST.
['[BOLD] Model', '[BOLD] E&M', '[BOLD] F&R']
Pre-training with augmented data generated by BT enhances the generalization ability of the model, thus we observe an improvement over the baseline. However, it does not introduce any new informal-to-formal transfer knowledge, leading to the least improvement among the three methods. In contrast, both F-Dis and M-Task introduce abundant transfer knowledge for FST. The augmented data of F-Dis includes various informal→formal rewrite knowledge derived from the MT models, allowing the model to better handle the test instances whose patterns are never seen in the original training data; while M-Task introduces GEC knowledge that helps improve formality in terms of grammar. We then combine all these beneficial augmented data for pre-training.