Parallel Data Augmentation for Formality Style Transfer
Table 1: The comparison of simultaneous training (ST) and Pre-train & Fine-tuning (PT&FT). Down-sampling and up-sampling are for balancing the size of the augmented data and the original data. Specifically, down-sampling samples augmented data, while up-sampling increases the frequency of the original data.
['[BOLD] Model', '[BOLD] E&M', '[BOLD] F&R']
ST with the augmented and original data leads to a performance decline, because the noisy augmented data cannot achieve desirable performance by itself and may distract the model from exploiting the original data in simultaneous training. In contrast, PT&FT only uses the augmented data in the pre-training phase and treats it as the prior knowledge supplementary to the original training data, reducing the negative effects of the augmented data and improving the results.