An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Table 6: Test results on eight BLUE tasks.
['Model', 'BlueBERT biomedical', 'BlueBERT clinical', 'MT-BioBERT Fine-Tune', 'MT-BlueBERT Fine-Tunebiomedical', 'MT-BlueBERT Fine-Tuneclinical']
Next, we also compare MT-BERT with its variants on all BLUE tasks. While confusing results were obtained when combing variety of tasks in both biomedical and clinical domains, we observed again that MTL models pretrained on biomedical literature perform better in biomedical tasks; and MTL models pretrained on both biomedical literature and clinical notes perform better in clinical tasks. These observations may suggest that it might be helpful to train separate deep neural networks on different types of text genres in BioNLP.