Evaluation of Sentence Embedding Models for Natural Language Understanding Problems in Russian
Table 5: ELMo unsupervised experiments results.
['Method', 'Domain', '[BOLD] MCQA Accuracy', '[BOLD] MCQA F1', '[BOLD] NSP Accuracy', '[BOLD] NSP F1', '[BOLD] PI Accuracy', '[BOLD] PI F1']
The performance of these methods is quite sensitive to the source of training data for ELMo model. For example, regarding the MCQA task, models trained on the News and Twitter corpora perform better than the model trained on Wikipedia, especially when logistic regression is used. In the unsupervised setting, the quality of next sentence prediction task highly depends on the source of training data ELMo model, too. However, in most cases there is no significant difference in results between three ELMo models.