Unsupervised Cross-lingual Representation Learning for Speech Recognition
Table 5: Impact of language similarity on cross-lingual transfer. We simulate a low-resource language scenario by using only 5 hours of Italian CommonVoice data and add 50 hours from another language for pretraining. We fine-tune on 1 hour of Italian supervised data.
['Model Number of pretraining hours', '[BOLD] #pt Number of pretraining hours', '[BOLD] #ft Number of pretraining hours', '[BOLD] it 5h (it)', '[BOLD] it 5h (it) + 50h ( )', '[BOLD] es 5h (it) + 50h ( )', '[BOLD] de 5h (it) + 50h ( )', '[BOLD] en 5h (it) + 50h ( )', '[BOLD] ru 5h (it) + 50h ( )', '[BOLD] ka 5h (it) + 50h ( )', '[BOLD] zh 5h (it) + 50h ( )']
We pretrain models on the 5h as well as 50h of unlabeled data from several other languages: Italian, Spanish, German, English, Russian, Kabyle and Chinese. Finally, we fine-tune each model on 1h of Italian labeled data. Distant languages, e.g., Kabyle or Chinese are less effective. In order to improve performance on a low-resource language, it is best to add unlabeled data from a closely-related language. However one could argue that Pashto and Kurmanji should be closer to each other since they are both Iranian languages. The purpose of this analysis is not to recover full language families but to better understand how our model allocates the latent representations across languages. Future work may investigate whether encouraging shared tokens between similar languages could further help cross-lingual transfer.