The SOFC-Exp Corpus and Neural Approachesto Information Extraction in the Materials Science Domain
Table 9: Experiments: modeling mention types in synthesis procedure data set. Results from Mysore et al. (2017) are not directly comparable to ours as they are based on a slightly different data set; our BiLSTM mat2vec+word2vec roughly corresponds to their BiLSTM-CRF model.
['[BOLD] Model', '[BOLD] micro-avg. F1']
Detailed scores by entity type can be found in the Supplementary Material. We chose to use the data split suggested by the authors for the NER task, using 200 documents for training, and 15 documents for each dev and test set. Among the non-BERT-based systems, the BiLSTM variant using both mat2vec and word2vec performs best, indicating that the two pre-trained embeddings contain complementary information with regard to this task. The best performance is reached by the BiLSTM model including word2vec, mat2vec, bpe and SciBERT embeddings, with 92.2 micro-average F1 providing a strong baseline for future work.