[
Table 3: Precision, recall, and F1 scores for different variations of our proposed model.
['Dataset', 'Metric', 'No pretrain', '[ITALIC] LMf pretrain', '[ITALIC] LMb pretrain', 'BiLM pretrain']
Our model gives an absolute improvement of around 2% and 0.5% in F1 score on NCBI-disease and BC5CDR dataset respectively over the model with no pretraining. We note that for all the datasets, LMf pretraining and LMb pretraining also gives an improvement over no pretraining. From the results, we also observe that the BiLM pretraining achieves better F1 score and precision in comparison to LMf pretraining and LMb pretraining, thus highlighting the importance of performing language modeling in both directions.