Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation
Table 8: Validation BLEU score (unless stated otherwise) obtained by freezing various parts of the mBART and of adding adapters for Xx → En. ‘ft’ refers to fine-tuning, i.e. unfreezing. We reproduce some rows from Table 3 for reference.
['[EMPTY]', '[BOLD] Vi-En', '[BOLD] It-En', '[BOLD] Si-En', '[BOLD] Ne-En', '[BOLD] My-En']
Across all language pairs we see improvements from fine-tuning layer norm parameters over not fine-tuning them, and additional improvements from adding adapters, indicating both forms of adding flexibility are useful.