Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Table 3: Average Prediction on Test set for Baseline BERT
['[EMPTY]', '[BOLD] 1', '[BOLD] 2', '[BOLD] 3', '[BOLD] 4', '[BOLD] 5', '[BOLD] Avg', '[BOLD] Chen1']
Our baseline model, using the SMM4H winning team’s model parameters but without their corpus for retraining, reported as ”BERT_noRetrained” Chen et al. We therefore ran the model multiple times and only included the results of the first five models with non-zero scores as our baseline. Notably, the F1-score is nearly identical to that reported by Chen et al Chen et al. Due to this convergence issue, we decreased our learning rate from 5e-5 to 2e-5 for our subsequent models.