Regularization Advantages of Multilingual Neural Language Models for Low Resource Domains
Table 2: Perplexity of the three LMs.
['[EMPTY]', 'Model', 'Number of words in training data 40K', 'Number of words in training data 100K', 'Number of words in training data 200K', 'Number of words in training data 300K', 'FULL']
As can be observed, mono-AWD-LSTM and mono-LSTM perform similarly weak in very low resource settings, while multi-AWD-LSTM outperforms both by a large margin in all four languages. When training data is sufficiently large, mono-AWD-LSTM achieves the best performance among all models, where multi-AWD-LSTM performs on par with or similar to it.