FinBERT: A Pretrained Language Model for Financial Communications
Table 3: Performance of pretraining on different financial corpus.
['[EMPTY]', 'PhraseBank', '10-Ks/10-Qs 0.835', 'Earnings Call 0.843', 'Analyst Reports 0.845', 'All 0.856']
Corpus Contribution We also train different FinBERT models on three financial corpus separately. It shows that FinBERT trained on all corpora achieves the overall best performance indicating that combining additional financial communication corpus could improve the language model quality. Among three datasets, Analyst Reports dataset appears to perform well in three different tasks, even though it only has 1.1 billion word tokens. Does it suggest that Analyst Reports data contains more information content than corporate reports and earnings call transcripts? We leave it for future research.