The SOFC-Exp Corpus and Neural Approachesto Information Extraction in the Materials Science Domain
Table 7: Experiments: slot identification. Model comparison in terms of macro F1.
['[BOLD] Model', '[BOLD] dev', '[BOLD] test']
As for entity typing, we train and evaluate our model on the subset of sentences marked as experiment-describing, which contain 4,263 slot instances. Again, the CRF baseline outperforms the BiLSTM when using only mat2vec and/or word2vec embeddings. The addition of BERT or SciBERT embeddings improves performance. However, on this task, the BiLSTM model with (Sci)BERT embeddings outperforms the fine-tuned original (Sci)BERT model. Compared to the other two tasks, this task requires more complex reasoning and has a larger number of possible output classes. We assume that in such a setting, adding more abstraction power to the model (in the form of a BiLSTM) leads to better results.