A Novel Neural Network Model for Joint POS Tagging and Graph-based Dependency Parsing
Table 2: Official macro-averaged LAS F1 scores of MQuni and baselines from the CoNLL 2017 shared task on UD parsing (Zeman et al., 2017): http://universaldependencies.org/conll17/results-las.html. “All” refers to the averaged score over all 81 test sets, which is used as the main metric for ranking participating systems. Big: the averaged score over 55/81 test sets whose training treebanks are big and have development data available. PUD: the averaged score over 14/81 test sets that are additional parallel ones, produced separately and their domain may be different from their training data. Sma.: the averaged score over 8/81 test sets whose training treebanks are small, i.e., they lack development data and some of them have very little training data. Sur.: the averaged score over 4/81 remaining test sets for surprise languages. Here the subscript denotes the official rank out of 33 participating systems. R-S is the system rank where the 4 surprise language test sets are not taken into account.
['[BOLD] System', '[BOLD] All', '[BOLD] Big', '[BOLD] PUD', '[BOLD] Sma.', '[BOLD] Sur.', '[BOLD] R-S']
We obtain 1% absolute higher averaged scores than the baseline UDPipe 1.1 Specifically, we obtain a highest rank at 8th place for the PUD category, showing that our parsing model jPTDP is particularly good when it is applied to a real practical application in out-of-domain data. Unlike the baseline UDPipe 1.1 and others, for each surprise language, we simply train a joint model just on the sample data of few sentences with gold-standard annotation provided before the test phase, i.e., we utilize neither external resources nor a cross-lingual technique nor a delexicalized parser. So, it is not surprising that we obtain a very low averaged score over the 4 surprise language test sets. When the 4 surprise language test sets are not taken into account, we obtain a rank in top-10 participating systems.