Cross-Lingual Natural Language Generation via Pre-Training
Table 10: Effects of different fine-tuning strategies. Dec, Enc and ET represent fine-tuning the parameters of the decoder, the encoder, and the Transformer layers of the encoder, respectively. Same shorthands apply as in Table 1.
['[EMPTY]', 'Supervised En-En-QG BL-4', 'Supervised En-En-QG MTR', 'Supervised En-En-QG RG-L', 'Zero-Shot Zh-Zh-QG BL-4', 'Zero-Shot Zh-Zh-QG MTR', 'Zero-Shot Zh-Zh-QG RG-L']
It can be observed that fine-tuning encoder parameters, our model obtain an impressive performance for both English and Chinese QG, which shows the strong cross-lingual transfer ability of our model. When fine-tuning all the parameters, the model achieves the best score for English QG, but it suffers a performance drop when evaluating on Chinese QG. We find that fine-tuning decoder hurts cross-lingual decoding, and the model learns to only decode English words. For only fine-tuning decoder, the performance degrades by a large margin for both languages because of the underfitting issue, which indicates the necessity of fine-tuning encoder.