From POS tagging to dependency parsing for biomedical event extraction
Table 11: Biomedical event extraction results. The subscripts denote results for which TEES is trained without the dependency labels.
['[BOLD] System', '[BOLD] System', '[BOLD] Development R', '[BOLD] Development P', '[BOLD] Development F1', '[BOLD] Test R', '[BOLD] Test P', '[BOLD] Test F1']
In this section, we present our empirical approach to evaluate different POS tagging and dependency parsing models on benchmark biomedical corpora. Their system used the Stanford-Biaffine parser (v2) trained on a dataset combining PTB, Brown corpus, and GENIA treebank data. The second row presents our score for the pre-trained BLLIP+Bio model; remaining rows show scores using re-trained parsing models. This is logical because the BioNLP 2009 shared task dataset was a subset of the GENIA corpus. Among the four dependency parsers trained on GENIA, Stanford-Biaffine, jPTDP and NLP4J-dep produce similar event extraction scores on the development set, while on the the test set jPTDP and NLP4J-dep obtain the lowest and highest scores, respectively. In most cases, compared to using dependency labels, event extraction scores drop on the development set (except NLP4J-dep trained on CRAFT), while they increase on the test set (except NLP4J-dep trained on GENIA and Stanford-NNdep trained on CRAFT). Without dependency labels, better event extraction scores on the development set corresponds to better scores on the test set. In addition, the differences in these event extraction scores without dependency labels are more consistent with the parsing performance differences than the scores with dependency labels.