Scientific Information Extraction with Semi-supervised Neural Tagging
Table 5: F1 score results on the test set for different categories: T indicates Task, P indicates Process, M is Material and K is Keyword identification (SubTask A). * is transductive model.
['Span Level', 'T', 'P', 'M', 'K']
The results show that our neural sequence tagging models significantly outperforms the state of the art and both baselines. It further shows that our system achieves significant boost from semi-supervised learning using unlabeled data. We observe significant improvement by using ULM+GraphInterp and ULM+GraphFeat over best SemEval and our best supervised system on all three categories at both token and span levels. We further observe that systems’ performance on Task classification is much lower than Process and Material. This is in part because Task is much less frequent than the other types. In addition, Task keyphrases often include verb phrases while the other two domains mainly consists of noun phrases. An analysis of confusion patterns show that the most frequent type confusions are between Process and Material. However, we observe that ULM+GraphFeat* can greatly reduce the confusion, with 3.5% relative improvement of Process and 3.6% relative improvement of Process over ULM+GraphInterp on token level.