An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Table 1: Summary of eight tasks in the BLUE benchmark. More details can be found in Peng et al. (2019).
['Corpus', 'Task', 'Metrics', 'Domain', 'Train', 'Dev', 'Test']
We evaluate the performance of the models on 8 datasets in the BLUE benchmark used by Peng et al. Briefly, ClinicalSTS is a corpus of sentence pairs selected from Mayo Clinics’s clinical data warehouse Wang et al. The i2b2 2010 dataset was collected from three different hospitals and was annotated by medical practitioners for eight types of relations between problems and treatments Uzuner et al. For a fair comparison, we use the same training, development and test sets to train and evaluate the models. ShARe /CLEF is a collection of 299 de-identified clinical free-text notes from the MIMIC-II database Suominen et al. This corpus is for disease entity recognition. There are two limitations to this work. First, our results on MTL training across all BLUE benchmark show that MTL is not always effective. We are interested in exploring further the characterization of task relationships. For example, it might be of interest to not using the BC5CDR corpus in the relation extraction task in future. Second, we studied one approach to MTL by sharing the encoder between all tasks while keeping several task-specific decoders. Liu et al.