Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Table 5: Average Prediction for BERTLARGE, BioBert, ClinicalBert and Max Ensemble Results with and without Preprocessor
['[ITALIC] No preprocessor', '[BOLD] BERT', '[BOLD] BioBERT', '[BOLD] ClinicalBERT', '[BOLD] Max Ensemble']
However, the difference in performance lessened considerably when the preprocessor was applied, with ClinicalBERT in particular observing improved predictions, with F1-score increasing from 0.58 to 0.62 and recall increasing from 0.55 to 0.61. Despite the poorer performance of BioBERT and ClinicalBERT, ensemble methods demonstrate that these alternative representations capture additional information which BERTLARGE does not.