FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
Table 4. Performance with different pre-training strategies
['Model', 'Loss', 'Accuracy', 'F1 Score']
We first measure the effect of further pre-training on the performance of the classifier. We compare three models: 1) No further pre-training (denoted by Vanilla BERT), 2) Further pre-training on classification training set (denoted by FinBERT-task), 3) Further pre-training on domain corpus, TRC2-financial (denoted by FinBERT-domain). Models are evaluated with loss, accuracy and macro average F1 scores on the test dataset.