Natural Language Premise Selection: Finding Supporting Statements for Mathematical Text
Table 5: Results for BERT and SciBERT.
['Model', 'MAP']
Even though BERT is not pre-trained using a mathematical corpus, it performs better than TF-IDF and PV-DBOW. SciBERT perform slightly better than BERT, since it was trained in a scientific corpus, but not in a mathematical corpus. This hints that BERT trained from scratch in a mathematical corpus could have even better results, however, this is outside the scope of this work.