A Weakly-Supervised Attention-based Visualization Tool for Assessing Political Affiliation
Table 1: Mean metrics for all the models
['[EMPTY]', 'Accuracy', 'F1-score']
For the network with Elmo embeddings, accuracy, recall, precision and F1-scores computed on the test set are calculated with both the smaller dataset and the larger dataset. Similar experiments were conducted on the network with static embeddings to assess how well it performs with limited data. As a result of the class imbalance in our training data, it is critical to evaluate performance using all of the metrics above. It is noteworthy to point out that in this regard, judging by the F1-score, the contextual embeddings provided by Elmo seems to perform marginally better than all of the static embeddings. It is also worth noting here that the performance of the network with Elmo embeddings is much more consistent when it is trained on the larger dataset. The threshold selected for class probabilities during classification while evaluating the above metrics was 0.5.