SciBert: A Pretrained Language Model for Scientific Text
Table 2: Comparing SciBert with the reported BioBert results on biomedical datasets.
['Task', 'Dataset', 'BioBert', 'SciBert']
Interesting, SciBert outperforms BioBert results on BC5CDR and ChemProt, and performs similarly on JNLPBA despite being trained on a substantially smaller biomedical corpus.